Image de couverture de livre

Does a Global Perspective Help Prune Sparse MoEs Elegantly?

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zhang, Zeliang, Ghosh, Nikhil, Liu, Jiani, Yu, Bin, Liu, Xiaodong
Format:	Preprint
Publié:	2026
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Diversifying the Expert Knowledge for Task-Agnostic Pruning in Sparse Mixture-of-Experts
par: Zhang, Zeliang, et autres
Publié: (2024)

Condense, Don't Just Prune: Enhancing Efficiency and Performance in MoE Layer Pruning
par: Cao, Mingyu, et autres
Publié: (2024)

Grove MoE: Towards Efficient and Superior MoE LLMs with Adjugate Experts
par: Wu, Haoyuan, et autres
Publié: (2025)

STUN: Structured-Then-Unstructured Pruning for Scalable MoE Pruning
par: Lee, Jaeseong, et autres
Publié: (2024)

LLaDA-MoE: A Sparse MoE Diffusion Language Model
par: Zhu, Fengqi, et autres
Publié: (2025)

Mix-MoE: Improving Multilingual Machine Translation of Large Language Models through Mixed MoEs
par: Li, Bo, et autres
Publié: (2026)

$μ$-MoE: Test-Time Pruning as Micro-Grained Mixture-of-Experts
par: Koike-Akino, Toshiaki, et autres
Publié: (2025)

Pangu Ultra MoE: How to Train Your Big MoE on Ascend NPUs
par: Tang, Yehui, et autres
Publié: (2025)

Why Instruction-Based Unlearning Fails in Diffusion Models?
par: Zhang, Zeliang, et autres
Publié: (2026)

SlimQwen: Exploring the Pruning and Distillation in Large MoE Model Pre-training
par: Tang, Shengkun, et autres
Publié: (2026)

MergeME: Model Merging Techniques for Homogeneous and Heterogeneous MoEs
par: Zhou, Yuhang, et autres
Publié: (2025)

Polysemantic Experts, Monosemantic Paths: Routing as Control in MoEs
par: Ye, Charles, et autres
Publié: (2026)

Continual Pre-training of MoEs: How robust is your router?
par: Thérien, Benjamin, et autres
Publié: (2025)

GRIN: GRadient-INformed MoE
par: Liu, Liyuan, et autres
Publié: (2024)

S2MoE: Robust Sparse Mixture of Experts via Stochastic Learning
par: Do, Giang, et autres
Publié: (2025)

GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMs
par: Deng, Jianing, et autres
Publié: (2026)

MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs
par: Xia, Xinfeng, et autres
Publié: (2025)

MoE-Pruner: Pruning Mixture-of-Experts Large Language Model using the Hints from Its Router
par: Xie, Yanyue, et autres
Publié: (2024)

Training Report of TeleChat3-MoE
par: Liu, Xinzhang, et autres
Publié: (2025)

$\texttt{MoE-RBench}$: Towards Building Reliable Language Models with Sparse Mixture-of-Experts
par: Chen, Guanjie, et autres
Publié: (2024)

EPS-MoE: Expert Pipeline Scheduler for Cost-Efficient MoE Inference
par: Qian, Yulei, et autres
Publié: (2024)

When Does Sparse MoE Help in Vision? The Role of Backbone Compute Leverage in Sparse Routing
par: Sun, Libo, et autres
Publié: (2026)

SEER-MoE: Sparse Expert Efficiency through Regularization for Mixture-of-Experts
par: Muzio, Alexandre, et autres
Publié: (2024)

Aligning Multimodal Sequential Recommendations via Robust Direct Preference Optimization with Sparse MoE
par: Huang, Hejin, et autres
Publié: (2026)

UniMoE-Audio: Unified Speech and Music Generation with Dynamic-Capacity MoE
par: Liu, Zhenyu, et autres
Publié: (2025)

State-of-the-Art Arabic Language Modeling with Sparse MoE Fine-Tuning and Chain-of-Thought Distillation
par: Singh, Navan Preet, et autres
Publié: (2026)

MoE-DiffuSeq: Enhancing Long-Document Diffusion Models with Sparse Attention and Mixture of Experts
par: Christoforos, Alexandros, et autres
Publié: (2025)

Ada-K Routing: Boosting the Efficiency of MoE-based LLMs
par: Yue, Tongtian, et autres
Publié: (2024)

Uni-MoE-2.0-Omni: Scaling Language-Centric Omnimodal Large Model with Advanced MoE, Training and Data
par: Li, Yunxin, et autres
Publié: (2025)

OmniMoE: An Efficient MoE by Orchestrating Atomic Experts at Scale
par: Shi, Jingze, et autres
Publié: (2026)

PLoP: Precise LoRA Placement for Efficient Finetuning of Large Models
par: Hayou, Soufiane, et autres
Publié: (2025)

Pangu Pro MoE: Mixture of Grouped Experts for Efficient Sparsity
par: Tang, Yehui, et autres
Publié: (2025)

MoEs Are Stronger than You Think: Hyper-Parallel Inference Scaling with RoE
par: Zibakhsh, Soheil, et autres
Publié: (2025)

Octavius: Mitigating Task Interference in MLLMs via LoRA-MoE
par: Chen, Zeren, et autres
Publié: (2023)

MoE-Sieve: Routing-Guided LoRA for Efficient MoE Fine-Tuning
par: Manzoni, Andrea
Publié: (2026)

Symbiotic-MoE: Unlocking the Synergy between Generation and Understanding
par: Liu, Xiangyue, et autres
Publié: (2026)

Sigma-MoE-Tiny Technical Report
par: Hu, Qingguo, et autres
Publié: (2025)

SparseLLM: Towards Global Pruning for Pre-trained Language Models
par: Bai, Guangji, et autres
Publié: (2024)

LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Training
par: Qu, Xiaoye, et autres
Publié: (2024)

PMoL: Parameter Efficient MoE for Preference Mixing of LLM Alignment
par: Liu, Dongxu, et autres
Publié: (2024)