Cover Image

Chain-of-Experts: Unlocking the Communication Power of Mixture-of-Experts Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Zihan, Pan, Rui, Yao, Jiarui, Csordas, Robert, Li, Linjie, Yin, Lu, Wu, Jiajun, Zhang, Tong, Li, Manling, Liu, Shiwei
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Rewiring Experts on the Fly:Continuous Rerouting for Better Online Adaptation in Mixture-of-Expert models
by: Su, Guinan, et al.
Published: (2025)

SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention
by: Csordás, Róbert, et al.
Published: (2023)

Mixture of Sparse Attention: Content-Based Learnable Sparse Attention via Expert-Choice Routing
by: Piękos, Piotr, et al.
Published: (2025)

Group then Scale: Dynamic Mixture-of-Experts Multilingual Language Model
by: Li, Chong, et al.
Published: (2025)

Mixture of insighTful Experts (MoTE): The Synergy of Thought Chains and Expert Mixtures in Self-Alignment
by: Liu, Zhili, et al.
Published: (2024)

Cluster-Driven Expert Pruning for Mixture-of-Experts Large Language Models
by: Guo, Hongcheng, et al.
Published: (2025)

Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
by: Lu, Xudong, et al.
Published: (2024)

Understanding and Leveraging the Expert Specialization of Context Faithfulness in Mixture-of-Experts LLMs
by: Bai, Jun, et al.
Published: (2025)

ReXMoE: Reusing Experts with Minimal Overhead in Mixture-of-Experts
by: Tan, Zheyue, et al.
Published: (2025)

Router Upcycling: Leveraging Mixture-of-Routers in Mixture-of-Experts Upcycling
by: Ran, Junfeng, et al.
Published: (2025)

Scalable Model Editing via Customized Expert Networks
by: Yao, Zihan, et al.
Published: (2024)

XFT: Unlocking the Power of Code Instruction Tuning by Simply Merging Upcycled Mixture-of-Experts
by: Ding, Yifeng, et al.
Published: (2024)

Unveiling Super Experts in Mixture-of-Experts Large Language Models
by: Su, Zunhai, et al.
Published: (2025)

Layerwise Recurrent Router for Mixture-of-Experts
by: Qiu, Zihan, et al.
Published: (2024)

Chain-of-Action: Faithful and Multimodal Question Answering through Large Language Models
by: Pan, Zhenyu, et al.
Published: (2024)

Maximum Score Routing For Mixture-of-Experts
by: Dong, Bowen, et al.
Published: (2025)

A Closer Look into Mixture-of-Experts in Large Language Models
by: Lo, Ka Man, et al.
Published: (2024)

MoPE: Mixture of Prefix Experts for Zero-Shot Dialogue State Tracking
by: Tang, Tianwen, et al.
Published: (2024)

Mixture of Lookup Experts
by: Jie, Shibo, et al.
Published: (2025)

Elastic MoE: Unlocking the Inference-Time Scalability of Mixture-of-Experts
by: Gu, Naibin, et al.
Published: (2025)

DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
by: Dai, Damai, et al.
Published: (2024)

Upcycling Large Language Models into Mixture of Experts
by: He, Ethan, et al.
Published: (2024)

MoBiLE: Efficient Mixture-of-Experts Inference on Consumer GPU with Mixture of Big Little Experts
by: Zhao, Yushu, et al.
Published: (2025)

Scalable Training of Mixture-of-Experts Models with Megatron Core
by: Yan, Zijie, et al.
Published: (2026)

SciDFM: A Large Language Model with Mixture-of-Experts for Science
by: Sun, Liangtai, et al.
Published: (2024)

Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models
by: Qiu, Zihan, et al.
Published: (2025)

MEMoE: Enhancing Model Editing with Mixture of Experts Adaptors
by: Wang, Renzhi, et al.
Published: (2024)

Mixture Compressor for Mixture-of-Experts LLMs Gains More
by: Huang, Wei, et al.
Published: (2024)

Exploiting Mixture-of-Experts Redundancy Unlocks Multimodal Generative Abilities
by: Dutt, Raman, et al.
Published: (2025)

Conv-CoA: Improving Open-domain Question Answering in Large Language Models via Conversational Chain-of-Action
by: Pan, Zhenyu, et al.
Published: (2024)

GatePro: Parameter-Free Expert Selection Optimization for Mixture-of-Experts Models
by: Zheng, Chen, et al.
Published: (2025)

DiEP: Adaptive Mixture-of-Experts Compression through Differentiable Expert Pruning
by: Bai, Sikai, et al.
Published: (2025)

Mixture of Neuron Experts
by: Cheng, Runxi, et al.
Published: (2025)

Preserving Long-Tailed Expert Information in Mixture-of-Experts Tuning
by: He, Haoze, et al.
Published: (2026)

Modality Interactive Mixture-of-Experts for Fake News Detection
by: Liu, Yifan, et al.
Published: (2025)

Mixture of LoRA Experts
by: Wu, Xun, et al.
Published: (2024)

ExpertFlow: Efficient Mixture-of-Experts Inference via Predictive Expert Caching and Token Scheduling
by: He, Xin, et al.
Published: (2024)

MoEUT: Mixture-of-Experts Universal Transformers
by: Csordás, Róbert, et al.
Published: (2024)

Your Mixture-of-Experts LLM Is Secretly an Embedding Model For Free
by: Li, Ziyue, et al.
Published: (2024)

Multi-Head Mixture-of-Experts
by: Wu, Xun, et al.
Published: (2024)