Mixture-of-Experts as Soft Clustering: A Dual Jacobian-PCA Spectral Geometry Perspective

Documenti analoghi

• Rotary Positional Embeddings as Phase Modulation: Theoretical Bounds on the RoPE Base for Long-Context Transformers
  di: Liu, Feilong
  Pubblicazione: (2026)
• Routing-Free Mixture-of-Experts
  di: Liu, Yilun, et al.
  Pubblicazione: (2026)
• SEUF: Is Unlearning One Expert Enough for Mixture-of-Experts LLMs?
  di: Zhuang, Haomin, et al.
  Pubblicazione: (2024)
• Mixture of Heterogeneous Grouped Experts for Language Modeling
  di: Ma, Zhicheng, et al.
  Pubblicazione: (2026)
• Multi-Head Mixture-of-Experts
  di: Wu, Xun, et al.
  Pubblicazione: (2024)