MoE-I$^2$: Compressing Mixture of Experts Models through Inter-Expert Pruning and Intra-Expert Low-Rank Decomposition

Documenti analoghi

• MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts
  di: Jin, Peng, et al.
  Pubblicazione: (2024)
• MoE Pathfinder: Trajectory-driven Expert Pruning
  di: Yang, Xican, et al.
  Pubblicazione: (2025)
• Linear-MoE: Linear Sequence Modeling Meets Mixture-of-Experts
  di: Sun, Weigao, et al.
  Pubblicazione: (2025)
• $\infty$-MoE: Generalizing Mixture of Experts to Infinite Experts
  di: Takashiro, Shota, et al.
  Pubblicazione: (2026)
• EAC-MoE: Expert-Selection Aware Compressor for Mixture-of-Experts Large Language Models
  di: Chen, Yuanteng, et al.
  Pubblicazione: (2025)