Image de couverture de livre

AnyExperts: On-Demand Expert Allocation for Multimodal Language Models with Mixture of Expert

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Gao, Yuting, Lan, Wang, Zhao, Hengyuan, Huang, Linjiang, Liu, Si, Guo, Qingpei
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

OrdMoE: Preference Alignment via Hierarchical Expert Group Ranking in Multimodal Mixture-of-Experts LLMs
par: Gao, Yuting, et autres
Publié: (2025)

Optimal Expert-Attention Allocation in Mixture-of-Experts: A Scalable Law for Dynamic Model Design
par: Li, Junzhuo, et autres
Publié: (2026)

MC#: Mixture Compressor for Mixture-of-Experts Large Models
par: Huang, Wei, et autres
Publié: (2025)

Efficiently Editing Mixture-of-Experts Models with Compressed Experts
par: He, Yifei, et autres
Publié: (2025)

Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
par: Lu, Xudong, et autres
Publié: (2024)

Mixture of Diverse Size Experts
par: Sun, Manxi, et autres
Publié: (2024)

Mixture of Experts in Large Language Models
par: Zhang, Danyang, et autres
Publié: (2025)

Accelerating Mixture-of-Expert Inference with Adaptive Expert Split Mechanism
par: Yan, Jiaming, et autres
Publié: (2025)

Alloc-MoE: Budget-Aware Expert Activation Allocation for Efficient Mixture-of-Experts Inference
par: Liu, Baihui, et autres
Publié: (2026)

Mixture of Heterogeneous Grouped Experts for Language Modeling
par: Ma, Zhicheng, et autres
Publié: (2026)

EAC-MoE: Expert-Selection Aware Compressor for Mixture-of-Experts Large Language Models
par: Chen, Yuanteng, et autres
Publié: (2025)

Speculating Experts Accelerates Inference for Mixture-of-Experts
par: Madan, Vivan, et autres
Publié: (2026)

Expert Upcycling: Shifting the Compute-Efficient Frontier of Mixture-of-Experts
par: Dwivedi, Chaitanya, et autres
Publié: (2026)

SEUF: Is Unlearning One Expert Enough for Mixture-of-Experts LLMs?
par: Zhuang, Haomin, et autres
Publié: (2024)

MoDE: A Mixture-of-Experts Model with Mutual Distillation among the Experts
par: Xie, Zhitian, et autres
Publié: (2024)

Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference
par: Chu, Kexin, et autres
Publié: (2025)

Modeling Expert Interactions in Sparse Mixture of Experts via Graph Structures
par: Nguyen-Nhat, Minh-Khoi, et autres
Publié: (2025)

Not All Models Suit Expert Offloading: On Local Routing Consistency of Mixture-of-Expert Models
par: Liang, Jingcong, et autres
Publié: (2025)

The Expert Strikes Back: Interpreting Mixture-of-Experts Language Models at Expert Level
par: Herbst, Jeremy, et autres
Publié: (2026)

UniPool: A Globally Shared Expert Pool for Mixture-of-Experts
par: Huang, Minbin, et autres
Publié: (2026)

Mixture of Raytraced Experts
par: Perin, Andrea, et autres
Publié: (2025)

HyperMoE: Towards Better Mixture of Experts via Transferring Among Experts
par: Zhao, Hao, et autres
Publié: (2024)

LightMoE: Reducing Mixture-of-Experts Redundancy through Expert Replacing
par: Hao, Jiawei, et autres
Publié: (2026)

Multi-Head Mixture-of-Experts
par: Wu, Xun, et autres
Publié: (2024)

Upcycling Large Language Models into Mixture of Experts
par: He, Ethan, et autres
Publié: (2024)

Dynamic Adaptive Shared Experts with Grouped Multi-Head Attention Mixture of Experts
par: Li, Cheng, et autres
Publié: (2025)

MoE-I$^2$: Compressing Mixture of Experts Models through Inter-Expert Pruning and Intra-Expert Low-Rank Decomposition
par: Yang, Cheng, et autres
Publié: (2024)

HELLoRA: Hot Experts Layer-Level Low-Rank Adaptation for Mixture-of-Experts Models
par: Wei, Jia, et autres
Publié: (2026)

Every Expert Matters: Towards Effective Knowledge Distillation for Mixture-of-Experts Language Models
par: Kim, Gyeongman, et autres
Publié: (2025)

PuzzleMoE: Efficient Compression of Large Mixture-of-Experts Models via Sparse Expert Merging and Bit-packed inference
par: Zhao, Yushu, et autres
Publié: (2025)

Understanding Expert Structures on Minimax Parameter Estimation in Contaminated Mixture of Experts
par: Yan, Fanqi, et autres
Publié: (2024)

MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts
par: Jin, Peng, et autres
Publié: (2024)

Exploring Expert Specialization through Unsupervised Training in Sparse Mixture of Experts
par: Nikolic, Strahinja, et autres
Publié: (2025)

Least-Loaded Expert Parallelism: Load Balancing An Imbalanced Mixture-of-Experts
par: Nguyen, Xuan-Phi, et autres
Publié: (2026)

Parameter-Efficient Routed Fine-Tuning: Mixture-of-Experts Demands Mixture of Adaptation Modules
par: Liu, Yilun, et autres
Publié: (2025)

Sparsity and Superposition in Mixture of Experts
par: Chaudhari, Marmik, et autres
Publié: (2025)

Mixture of Concept Bottleneck Experts
par: De Santis, Francesco, et autres
Publié: (2026)

Mixture of A Million Experts
par: He, Xu Owen
Publié: (2024)

Klotski: Efficient Mixture-of-Expert Inference via Expert-Aware Multi-Batch Pipeline
par: Fang, Zhiyuan, et autres
Publié: (2025)

SAME: Stabilized Mixture-of-Experts for Multimodal Continual Instruction Tuning
par: Xie, Zhen-Hao, et autres
Publié: (2026)