Upcycling Large Language Models into Mixture of Experts

Documenti analoghi

• Scalable Training of Mixture-of-Experts Models with Megatron Core
  di: Yan, Zijie, et al.
  Pubblicazione: (2026)
• Llama 3 Meets MoE: Efficient Upcycling
  di: Vavre, Aditya, et al.
  Pubblicazione: (2024)
• Reuse, Don't Retrain: A Recipe for Continued Pretraining of Language Models
  di: Parmar, Jupinder, et al.
  Pubblicazione: (2024)
• Router Upcycling: Leveraging Mixture-of-Routers in Mixture-of-Experts Upcycling
  di: Ran, Junfeng, et al.
  Pubblicazione: (2025)
• AceMath: Advancing Frontier Math Reasoning with Post-Training and Reward Modeling
  di: Liu, Zihan, et al.
  Pubblicazione: (2024)