Diversifying the Expert Knowledge for Task-Agnostic Pruning in Sparse Mixture-of-Experts

Documenti analoghi

• Training Large Reasoning Models Efficiently via Progressive Thought Encoding
  di: Zhang, Zeliang, et al.
  Pubblicazione: (2026)
• Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
  di: Lu, Xudong, et al.
  Pubblicazione: (2024)
• Routers Learn the Geometry of Their Experts: Geometric Coupling in Sparse Mixture-of-Experts
  di: Ahrac, Sagi, et al.
  Pubblicazione: (2026)
• SEER-MoE: Sparse Expert Efficiency through Regularization for Mixture-of-Experts
  di: Muzio, Alexandre, et al.
  Pubblicazione: (2024)
• CoSMoEs: Compact Sparse Mixture of Experts
  di: Huber, Patrick, et al.
  Pubblicazione: (2025)