Image de couverture de livre

The Illusion of Specialization: Unveiling the Domain-Invariant "Standing Committee" in Mixture-of-Experts Models

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Wang, Yan, Xu, Yitao, Shen, Nanhan, Su, Jinyan, Huang, Jimin, Zhu, Zining
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Unveiling Hidden Collaboration within Mixture-of-Experts in Large Language Models
par: Tang, Yuanbo, et autres
Publié: (2025)

MIDG: Mixture of Invariant Experts with knowledge injection for Domain Generalization in Multimodal Sentiment Analysis
par: Li, Yangle, et autres
Publié: (2025)

Accelerating Mixture-of-Expert Inference with Adaptive Expert Split Mechanism
par: Yan, Jiaming, et autres
Publié: (2025)

Exploring Expert Specialization through Unsupervised Training in Sparse Mixture of Experts
par: Nikolic, Strahinja, et autres
Publié: (2025)

MC#: Mixture Compressor for Mixture-of-Experts Large Models
par: Huang, Wei, et autres
Publié: (2025)

How Many Experts Are Enough? Towards Optimal Semantic Specialization for Mixture-of-Experts
par: Park, Sumin, et autres
Publié: (2025)

MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts
par: Jin, Peng, et autres
Publié: (2024)

AnyExperts: On-Demand Expert Allocation for Multimodal Language Models with Mixture of Expert
par: Gao, Yuting, et autres
Publié: (2025)

PWC-MoE: Privacy-Aware Wireless Collaborative Mixture of Experts
par: Su, Yang, et autres
Publié: (2025)

Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
par: Lu, Xudong, et autres
Publié: (2024)

Distributed Interpretability and Control for Large Language Models
par: Desai, Dev Arpan, et autres
Publié: (2026)

Mixture of A Million Experts
par: He, Xu Owen
Publié: (2024)

Toward Structural Multimodal Representations: Specialization, Selection, and Sparsification via Mixture-of-Experts
par: Choi, Hahyeon, et autres
Publié: (2026)

MoSE: Unveiling Structural Patterns in Graphs via Mixture of Subgraph Experts
par: Ye, Junda, et autres
Publié: (2025)

MoFE-Time: Mixture of Frequency Domain Experts for Time-Series Forecasting Models
par: Liu, Yiwen, et autres
Publié: (2025)

Each Rank Could be an Expert: Single-Ranked Mixture of Experts LoRA for Multi-Task Learning
par: Zhao, Ziyu, et autres
Publié: (2025)

Efficiently Editing Mixture-of-Experts Models with Compressed Experts
par: He, Yifei, et autres
Publié: (2025)

Nexus: Specialization meets Adaptability for Efficiently Training Mixture of Experts
par: Gritsch, Nikolas, et autres
Publié: (2024)

Unified Class and Domain Incremental Learning with Mixture of Experts for Indoor Localization
par: Singampalli, Akhil, et autres
Publié: (2025)

Learning from Streaming Data when Users Choose
par: Su, Jinyan, et autres
Publié: (2024)

Mixture of Experts in Large Language Models
par: Zhang, Danyang, et autres
Publié: (2025)

Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference
par: Chu, Kexin, et autres
Publié: (2025)

Feature-Guided SAE Steering for Refusal-Rate Control using Contrasting Prompts
par: Bhargav, Samaksh, et autres
Publié: (2025)

Understanding Expert Structures on Minimax Parameter Estimation in Contaminated Mixture of Experts
par: Yan, Fanqi, et autres
Publié: (2024)

Wisdom of Committee: Distilling from Foundation Model to Specialized Application Model
par: Liu, Zichang, et autres
Publié: (2024)

Mixture of Latent Experts Using Tensor Products
par: Su, Zhan, et autres
Publié: (2024)

Expert Upcycling: Shifting the Compute-Efficient Frontier of Mixture-of-Experts
par: Dwivedi, Chaitanya, et autres
Publié: (2026)

Routing Mamba: Scaling State Space Models with Mixture-of-Experts Projection
par: Zhan, Zheng, et autres
Publié: (2025)

LightMoE: Reducing Mixture-of-Experts Redundancy through Expert Replacing
par: Hao, Jiawei, et autres
Publié: (2026)

Input Domain Aware MoE: Decoupling Routing Decisions from Task Optimization in Mixture of Experts
par: Hua, Yongxiang, et autres
Publié: (2025)

Modeling Expert Interactions in Sparse Mixture of Experts via Graph Structures
par: Nguyen-Nhat, Minh-Khoi, et autres
Publié: (2025)

Mixture of Raytraced Experts
par: Perin, Andrea, et autres
Publié: (2025)

Least-Loaded Expert Parallelism: Load Balancing An Imbalanced Mixture-of-Experts
par: Nguyen, Xuan-Phi, et autres
Publié: (2026)

Multi-Task Vehicle Routing Solver via Mixture of Specialized Experts under State-Decomposable MDP
par: Pan, Yuxin, et autres
Publié: (2025)

Mixture of Heterogeneous Grouped Experts for Language Modeling
par: Ma, Zhicheng, et autres
Publié: (2026)

Mixture of Experts in a Mixture of RL settings
par: Willi, Timon, et autres
Publié: (2024)

UniPool: A Globally Shared Expert Pool for Mixture-of-Experts
par: Huang, Minbin, et autres
Publié: (2026)

Speculating Experts Accelerates Inference for Mixture-of-Experts
par: Madan, Vivan, et autres
Publié: (2026)

Not All Models Suit Expert Offloading: On Local Routing Consistency of Mixture-of-Expert Models
par: Liang, Jingcong, et autres
Publié: (2025)

Towards Efficient Pareto Set Approximation via Mixture of Experts Based Model Fusion
par: Tang, Anke, et autres
Publié: (2024)