Imagen de Portada

DiEP: Adaptive Mixture-of-Experts Compression through Differentiable Expert Pruning

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Bai, Sikai, Li, Haoxi, Zhang, Jie, Hong, Zicong, Guo, Song
Formato:	Preprint
Publicado:	2025
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Cluster-Driven Expert Pruning for Mixture-of-Experts Large Language Models
por: Guo, Hongcheng, et al.
Publicado: (2025)

Dropping Experts, Recombining Neurons: Retraining-Free Pruning for Sparse Mixture-of-Experts LLMs
por: Zhou, Yixiao, et al.
Publicado: (2025)

Diversifying the Expert Knowledge for Task-Agnostic Pruning in Sparse Mixture-of-Experts
por: Zhang, Zeliang, et al.
Publicado: (2024)

Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
por: Lu, Xudong, et al.
Publicado: (2024)

Mixture of Lookup Experts
por: Jie, Shibo, et al.
Publicado: (2025)

ToMoE: Converting Dense Large Language Models to Mixture-of-Experts through Dynamic Structural Pruning
por: Gao, Shangqian, et al.
Publicado: (2025)

Understanding and Leveraging the Expert Specialization of Context Faithfulness in Mixture-of-Experts LLMs
por: Bai, Jun, et al.
Publicado: (2025)

Pruning and Distilling Mixture-of-Experts into Dense Language Models
por: Kim, Junhyuck, et al.
Publicado: (2026)

Unveiling Super Experts in Mixture-of-Experts Large Language Models
por: Su, Zunhai, et al.
Publicado: (2025)

Layerwise Recurrent Router for Mixture-of-Experts
por: Qiu, Zihan, et al.
Publicado: (2024)

SEUF: Is Unlearning One Expert Enough for Mixture-of-Experts LLMs?
por: Zhuang, Haomin, et al.
Publicado: (2024)

Mixture of insighTful Experts (MoTE): The Synergy of Thought Chains and Expert Mixtures in Self-Alignment
por: Liu, Zhili, et al.
Publicado: (2024)

Mixture of Neuron Experts
por: Cheng, Runxi, et al.
Publicado: (2025)

EvoMoE: Expert Evolution in Mixture of Experts for Multimodal Large Language Models
por: Jing, Linglin, et al.
Publicado: (2025)

Chain-of-Experts: Unlocking the Communication Power of Mixture-of-Experts Models
por: Wang, Zihan, et al.
Publicado: (2025)

SEER-MoE: Sparse Expert Efficiency through Regularization for Mixture-of-Experts
por: Muzio, Alexandre, et al.
Publicado: (2024)

ReXMoE: Reusing Experts with Minimal Overhead in Mixture-of-Experts
por: Tan, Zheyue, et al.
Publicado: (2025)

Domain-Specific Pruning of Large Mixture-of-Experts Models with Few-shot Demonstrations
por: Dong, Zican, et al.
Publicado: (2025)

CoRE: Enhancing Metacognition with Label-free Self-evaluation in LRMs
por: Li, Haoxi, et al.
Publicado: (2025)

Router Upcycling: Leveraging Mixture-of-Routers in Mixture-of-Experts Upcycling
por: Ran, Junfeng, et al.
Publicado: (2025)

Preserving Long-Tailed Expert Information in Mixture-of-Experts Tuning
por: He, Haoze, et al.
Publicado: (2026)

Routing by Analogy: kNN-Augmented Expert Assignment for Mixture-of-Experts
por: Lyu, Boxuan, et al.
Publicado: (2026)

Training Matryoshka Mixture-of-Experts for Elastic Inference-Time Expert Utilization
por: Wang, Yaoxiang, et al.
Publicado: (2025)

Mixture Compressor for Mixture-of-Experts LLMs Gains More
por: Huang, Wei, et al.
Publicado: (2024)

$μ$-MoE: Test-Time Pruning as Micro-Grained Mixture-of-Experts
por: Koike-Akino, Toshiaki, et al.
Publicado: (2025)

DEER: Disentangled Mixture of Experts with Instance-Adaptive Routing for Generalizable Machine-Generated Text Detection
por: Ma, Guoxin, et al.
Publicado: (2025)

Rewiring Experts on the Fly:Continuous Rerouting for Better Online Adaptation in Mixture-of-Expert models
por: Su, Guinan, et al.
Publicado: (2025)

Modeling Professionalism in Expert Questioning through Linguistic Differentiation
por: D'Agostino, Giulia, et al.
Publicado: (2025)

MoBiLE: Efficient Mixture-of-Experts Inference on Consumer GPU with Mixture of Big Little Experts
por: Zhao, Yushu, et al.
Publicado: (2025)

ExpertFlow: Efficient Mixture-of-Experts Inference via Predictive Expert Caching and Token Scheduling
por: He, Xin, et al.
Publicado: (2024)

Understanding Multilingualism in Mixture-of-Experts LLMs: Routing Mechanism, Expert Specialization, and Layerwise Steering
por: Chen, Yuxin, et al.
Publicado: (2026)

MoE-Pruner: Pruning Mixture-of-Experts Large Language Model using the Hints from Its Router
por: Xie, Yanyue, et al.
Publicado: (2024)

ExpertRAG: Efficient RAG with Mixture of Experts -- Optimizing Context Retrieval for Adaptive LLM Responses
por: Gumaan, Esmail
Publicado: (2025)

Coupling Experts and Routers in Mixture-of-Experts via an Auxiliary Loss
por: Lv, Ang, et al.
Publicado: (2025)

Shortcut-connected Expert Parallelism for Accelerating Mixture-of-Experts
por: Cai, Weilin, et al.
Publicado: (2024)

$\infty$-MoE: Generalizing Mixture of Experts to Infinite Experts
por: Takashiro, Shota, et al.
Publicado: (2026)

P-React: Synthesizing Topic-Adaptive Reactions of Personality Traits via Mixture of Specialized LoRA Experts
por: Dan, Yuhao, et al.
Publicado: (2024)

Mixture of Experts for Low-Resource LLMs
por: Joseph, Ori Bar, et al.
Publicado: (2026)

MoDES: Accelerating Mixture-of-Experts Multimodal Large Language Models via Dynamic Expert Skipping
por: Huang, Yushi, et al.
Publicado: (2025)

SAMoRA: Semantic-Aware Mixture of LoRA Experts for Task-Adaptive Learning
por: Shi, Boyan, et al.
Publicado: (2026)