Image de couverture de livre

Unveiling Hidden Collaboration within Mixture-of-Experts in Large Language Models

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Tang, Yuanbo, Tang, Yan, Zhang, Naifan, Chen, Meixuan, Li, Yang
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Dy-mer: An Explainable DNA Sequence Representation Scheme using Dictionary Learning
par: Peng, Zhiyuan, et autres
Publié: (2024)

Mixture of Experts in Large Language Models
par: Zhang, Danyang, et autres
Publié: (2025)

MC#: Mixture Compressor for Mixture-of-Experts Large Models
par: Huang, Wei, et autres
Publié: (2025)

Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
par: Lu, Xudong, et autres
Publié: (2024)

Upcycling Large Language Models into Mixture of Experts
par: He, Ethan, et autres
Publié: (2024)

Not All Models Suit Expert Offloading: On Local Routing Consistency of Mixture-of-Expert Models
par: Liang, Jingcong, et autres
Publié: (2025)

PWC-MoE: Privacy-Aware Wireless Collaborative Mixture of Experts
par: Su, Yang, et autres
Publié: (2025)

The Illusion of Specialization: Unveiling the Domain-Invariant "Standing Committee" in Mixture-of-Experts Models
par: Wang, Yan, et autres
Publié: (2026)

EAC-MoE: Expert-Selection Aware Compressor for Mixture-of-Experts Large Language Models
par: Chen, Yuanteng, et autres
Publié: (2025)

HELM: Hyperbolic Large Language Models via Mixture-of-Curvature Experts
par: He, Neil, et autres
Publié: (2025)

MoEQuant: Enhancing Quantization for Mixture-of-Experts Large Language Models via Expert-Balanced Sampling and Affinity Guidance
par: Hu, Xing, et autres
Publié: (2025)

MoSE: Unveiling Structural Patterns in Graphs via Mixture of Subgraph Experts
par: Ye, Junda, et autres
Publié: (2025)

Efficiently Editing Mixture-of-Experts Models with Compressed Experts
par: He, Yifei, et autres
Publié: (2025)

Drift-aware Collaborative Assistance Mixture of Experts for Heterogeneous Multistream Learning
par: Yu, En, et autres
Publié: (2025)

T-REX: Mixture-of-Rank-One-Experts with Semantic-aware Intuition for Multi-task Large Language Model Finetuning
par: Zhang, Rongyu, et autres
Publié: (2024)

FactorLLM: Factorizing Knowledge via Mixture of Experts for Large Language Models
par: Zhao, Zhongyu, et autres
Publié: (2024)

A Stronger Mixture of Low-Rank Experts for Fine-Tuning Foundation Models
par: Sun, Mengyang, et autres
Publié: (2025)

WDMoE: Wireless Distributed Large Language Models with Mixture of Experts
par: Xue, Nan, et autres
Publié: (2024)

Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models
par: Guo, Yongxin, et autres
Publié: (2024)

AnyExperts: On-Demand Expert Allocation for Multimodal Language Models with Mixture of Expert
par: Gao, Yuting, et autres
Publié: (2025)

Towards Efficient Pareto Set Approximation via Mixture of Experts Based Model Fusion
par: Tang, Anke, et autres
Publié: (2024)

FourierMoE: Fourier Mixture-of-Experts Adaptation of Large Language Models
par: Jiang, Juyong, et autres
Publié: (2026)

Efficient Training of Large-Scale AI Models Through Federated Mixture-of-Experts: A System-Level Approach
par: Chen, Xiaobing, et autres
Publié: (2025)

MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts
par: Jin, Peng, et autres
Publié: (2024)

Towards Faster Language Model Inference Using Mixture-of-Experts Flow Matching
par: Li, Aihua
Publié: (2026)

SMILE: Zero-Shot Sparse Mixture of Low-Rank Experts Construction From Pre-Trained Foundation Models
par: Tang, Anke, et autres
Publié: (2024)

Reasoning Language Model Inference Serving Unveiled: An Empirical Study
par: Li, Qi, et autres
Publié: (2025)

DeLo: Dual Decomposed Low-Rank Experts Collaboration for Continual Missing Modality Learning
par: Liu, Xiwei, et autres
Publié: (2026)

Mixtures of SubExperts for Large Language Continual Learning
par: Kang, Haeyong
Publié: (2025)

Accelerating Mixture-of-Expert Inference with Adaptive Expert Split Mechanism
par: Yan, Jiaming, et autres
Publié: (2025)

Every Expert Matters: Towards Effective Knowledge Distillation for Mixture-of-Experts Language Models
par: Kim, Gyeongman, et autres
Publié: (2025)

Towards Generalization-Oriented Models for Vehicle Routing Problems with Mixture-of-Experts
par: Miao, Changhao, et autres
Publié: (2026)

MoDE: A Mixture-of-Experts Model with Mutual Distillation among the Experts
par: Xie, Zhitian, et autres
Publié: (2024)

PuzzleMoE: Efficient Compression of Large Mixture-of-Experts Models via Sparse Expert Merging and Bit-packed inference
par: Zhao, Yushu, et autres
Publié: (2025)

Optimal Expert-Attention Allocation in Mixture-of-Experts: A Scalable Law for Dynamic Model Design
par: Li, Junzhuo, et autres
Publié: (2026)

Mixture of Heterogeneous Grouped Experts for Language Modeling
par: Ma, Zhicheng, et autres
Publié: (2026)

SAME: Stabilized Mixture-of-Experts for Multimodal Continual Instruction Tuning
par: Xie, Zhen-Hao, et autres
Publié: (2026)

Unveiling and Addressing Pseudo Forgetting in Large Language Models
par: Sun, Huashan, et autres
Publié: (2024)

HELLoRA: Hot Experts Layer-Level Low-Rank Adaptation for Mixture-of-Experts Models
par: Wei, Jia, et autres
Publié: (2026)

Dynamic Adaptive Shared Experts with Grouped Multi-Head Attention Mixture of Experts
par: Li, Cheng, et autres
Publié: (2025)