Pruning and Distilling Mixture-of-Experts into Dense Language Models

Documenti analoghi

• Every Expert Matters: Towards Effective Knowledge Distillation for Mixture-of-Experts Language Models
  di: Kim, Gyeongman, et al.
  Pubblicazione: (2025)
• PromptKD: Distilling Student-Friendly Knowledge for Generative Language Models via Prompt Tuning
  di: Kim, Gyeongman, et al.
  Pubblicazione: (2024)
• Bridging the Gap between Expert and Language Models: Concept-guided Chess Commentary Generation and Evaluation
  di: Kim, Jaechang, et al.
  Pubblicazione: (2024)
• Raon-Speech Technical Report
  di: Kim, Beomsoo, et al.
  Pubblicazione: (2026)
• Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
  di: Lu, Xudong, et al.
  Pubblicazione: (2024)