CartesianMoE: Boosting Knowledge Sharing among Experts via Cartesian Product Routing in Mixture-of-Experts

Documenti analoghi

• MaskMoE: Boosting Token-Level Learning via Routing Mask in Mixture-of-Experts
  di: Su, Zhenpeng, et al.
  Pubblicazione: (2024)
• DSMoE: Matrix-Partitioned Experts with Dynamic Routing for Computation-Efficient Dense LLMs
  di: Lv, Minxuan, et al.
  Pubblicazione: (2025)
• Finedeep: Mitigating Sparse Activation in Dense LLMs via Multi-Layer Fine-Grained Experts
  di: Pan, Leiyu, et al.
  Pubblicazione: (2025)
• MiLe Loss: a New Entropy-Weighed Loss for Mitigating the Bias of Learning Difficulties in Large Language Models
  di: Su, Zhenpeng, et al.
  Pubblicazione: (2023)
• UniAttn: Reducing Inference Costs via Softmax Unification for Post-Training LLMs
  di: Xiong, Yizhe, et al.
  Pubblicazione: (2025)