Millions of States: Designing a Scalable MoE Architecture with RWKV-7 Meta-learner

Documenti analoghi

• State Tuning: State-based Test-Time Scaling on RWKV-7
  di: Xiao, Liu, et al.
  Pubblicazione: (2025)
• Cross-attention for State-based model RWKV-7
  di: Xiao, Liu, et al.
  Pubblicazione: (2025)
• MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs
  di: Xia, Xinfeng, et al.
  Pubblicazione: (2025)
• STUN: Structured-Then-Unstructured Pruning for Scalable MoE Pruning
  di: Lee, Jaeseong, et al.
  Pubblicazione: (2024)
• X-MoE: Enabling Scalable Training for Emerging Mixture-of-Experts Architectures on HPC Platforms
  di: Yuan, Yueming, et al.
  Pubblicazione: (2025)