Cover Image

Not All Models Suit Expert Offloading: On Local Routing Consistency of Mixture-of-Expert Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liang, Jingcong, Wang, Siyuan, Tian, Miren, Li, Yitong, Tang, Duyu, Wei, Zhongyu
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

In-depth Analysis on Caching and Pre-fetching in Mixture of Experts Offloading
by: Lin, Shuning, et al.
Published: (2025)

Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
by: Lu, Xudong, et al.
Published: (2024)

HardcoreLogic: Challenging Large Reasoning Models with Long-tail Logic Puzzle Games
by: Liang, Jingcong, et al.
Published: (2025)

RASA: Routing-Aware Safety Alignment for Mixture-of-Experts Models
by: Liang, Jiacheng, et al.
Published: (2026)

Efficiently Editing Mixture-of-Experts Models with Compressed Experts
by: He, Yifei, et al.
Published: (2025)

Routing Mamba: Scaling State Space Models with Mixture-of-Experts Projection
by: Zhan, Zheng, et al.
Published: (2025)

Soft-to-Hard Routing in Sparse Mixture-of-Experts Models
by: Rastegar, Reza
Published: (2026)

Probing Semantic Routing in Large Mixture-of-Expert Models
by: Olson, Matthew Lyle, et al.
Published: (2025)

Routing-Free Mixture-of-Experts
by: Liu, Yilun, et al.
Published: (2026)

Multilingual Routing in Mixture-of-Experts
by: Bandarkar, Lucas, et al.
Published: (2025)

Towards Generalization-Oriented Models for Vehicle Routing Problems with Mixture-of-Experts
by: Miao, Changhao, et al.
Published: (2026)

PERFT: Parameter-Efficient Routed Fine-Tuning for Mixture-of-Expert Model
by: Liu, Yilun, et al.
Published: (2024)

AnyExperts: On-Demand Expert Allocation for Multimodal Language Models with Mixture of Expert
by: Gao, Yuting, et al.
Published: (2025)

Mixture of Lookup Experts
by: Jie, Shibo, et al.
Published: (2025)

RouteHijack: Routing-Aware Attack on Mixture-of-Experts LLMs
by: Xu, Zhiyuan, et al.
Published: (2026)

Optimal Expert-Attention Allocation in Mixture-of-Experts: A Scalable Law for Dynamic Model Design
by: Li, Junzhuo, et al.
Published: (2026)

MC#: Mixture Compressor for Mixture-of-Experts Large Models
by: Huang, Wei, et al.
Published: (2025)

Sparse Models, Sparse Safety: Unsafe Routes in Mixture-of-Experts LLMs
by: Jiang, Yukun, et al.
Published: (2026)

Unveiling Hidden Collaboration within Mixture-of-Experts in Large Language Models
by: Tang, Yuanbo, et al.
Published: (2025)

Each Rank Could be an Expert: Single-Ranked Mixture of Experts LoRA for Multi-Task Learning
by: Zhao, Ziyu, et al.
Published: (2025)

Mixture of Experts in Large Language Models
by: Zhang, Danyang, et al.
Published: (2025)

Neural Inhibition Improves Dynamic Routing and Mixture of Experts
by: Zou, Will Y., et al.
Published: (2025)

A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applications
by: Mu, Siyuan, et al.
Published: (2025)

HELLoRA: Hot Experts Layer-Level Low-Rank Adaptation for Mixture-of-Experts Models
by: Wei, Jia, et al.
Published: (2026)

Geometric Mixture-of-Experts with Curvature-Guided Adaptive Routing for Graph Representation Learning
by: Cao, Haifang, et al.
Published: (2026)

Modeling Expert Interactions in Sparse Mixture of Experts via Graph Structures
by: Nguyen-Nhat, Minh-Khoi, et al.
Published: (2025)

Mixture of Diverse Size Experts
by: Sun, Manxi, et al.
Published: (2024)

ProbMoE: Differentiable Probabilistic Routing for Mixture-of-Experts
by: Zhao, Heng, et al.
Published: (2026)

Variational Routing: A Scalable Bayesian Framework for Calibrated Mixture-of-Experts Transformers
by: Li, Albus Yizhuo, et al.
Published: (2026)

Theory on Mixture-of-Experts in Continual Learning
by: Li, Hongbo, et al.
Published: (2024)

MoDE: A Mixture-of-Experts Model with Mutual Distillation among the Experts
by: Xie, Zhitian, et al.
Published: (2024)

EAC-MoE: Expert-Selection Aware Compressor for Mixture-of-Experts Large Language Models
by: Chen, Yuanteng, et al.
Published: (2025)

L2R: Low-Rank and Lipschitz-Controlled Routing for Mixture-of-Experts
by: Yang, Minghao, et al.
Published: (2026)

Alloc-MoE: Budget-Aware Expert Activation Allocation for Efficient Mixture-of-Experts Inference
by: Liu, Baihui, et al.
Published: (2026)

FactorLLM: Factorizing Knowledge via Mixture of Experts for Large Language Models
by: Zhao, Zhongyu, et al.
Published: (2024)

Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models
by: Guo, Yongxin, et al.
Published: (2024)

MVMoE: Multi-Task Vehicle Routing Solver with Mixture-of-Experts
by: Zhou, Jianan, et al.
Published: (2024)

A Stronger Mixture of Low-Rank Experts for Fine-Tuning Foundation Models
by: Sun, Mengyang, et al.
Published: (2025)

Speculating Experts Accelerates Inference for Mixture-of-Experts
by: Madan, Vivan, et al.
Published: (2026)

Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference
by: Chu, Kexin, et al.
Published: (2025)