DTop-p MoE: Sparsity-Controlled Dynamic Top-p MoE for Foundation Model Pre-training

Documenti analoghi

• MoE-Infinity: Efficient MoE Inference on Personal Machines with Sparsity-Aware Expert Cache
  di: Xue, Leyang, et al.
  Pubblicazione: (2024)
• Sparsity is Combinatorial Depth: Quantifying MoE Expressivity via Tropical Geometry
  di: Su, Ye, et al.
  Pubblicazione: (2026)
• Pangu Pro MoE: Mixture of Grouped Experts for Efficient Sparsity
  di: Tang, Yehui, et al.
  Pubblicazione: (2025)
• MoE-Prefill: Zero Redundancy Overheads in MoE Prefill Serving
  di: Su, Zhaoyuan, et al.
  Pubblicazione: (2026)
• MoE-PHDS: One MoE checkpoint for flexible runtime sparsity
  di: Hannah, Lauren. A, et al.
  Pubblicazione: (2025)