Cover Image

Understanding and Leveraging the Expert Specialization of Context Faithfulness in Mixture-of-Experts LLMs

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Bai, Jun, Tong, Minghao, Liu, Yang, Jia, Zixia, Zheng, Zilong
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Understanding Multilingualism in Mixture-of-Experts LLMs: Routing Mechanism, Expert Specialization, and Layerwise Steering
by: Chen, Yuxin, et al.
Published: (2026)

Mixture of In-Context Experts Enhance LLMs' Long Context Awareness
by: Lin, Hongzhan, et al.
Published: (2024)

Router Upcycling: Leveraging Mixture-of-Routers in Mixture-of-Experts Upcycling
by: Ran, Junfeng, et al.
Published: (2025)

SEUF: Is Unlearning One Expert Enough for Mixture-of-Experts LLMs?
by: Zhuang, Haomin, et al.
Published: (2024)

ExpertSteer: Intervening in LLMs through Expert Knowledge
by: Wang, Weixuan, et al.
Published: (2025)

MoPE: Mixture of Prefix Experts for Zero-Shot Dialogue State Tracking
by: Tang, Tianwen, et al.
Published: (2024)

Towards Greater Leverage: Scaling Laws for Efficient Mixture-of-Experts Language Models
by: Tian, Changxin, et al.
Published: (2025)

TokenSwift: Lossless Acceleration of Ultra Long Sequence Generation
by: Wu, Tong, et al.
Published: (2025)

Understanding the Mixture-of-Experts with Nadaraya-Watson Kernel
by: Zheng, Chuanyang, et al.
Published: (2025)

Mixture of Experts for Low-Resource LLMs
by: Joseph, Ori Bar, et al.
Published: (2026)

Mixture Compressor for Mixture-of-Experts LLMs Gains More
by: Huang, Wei, et al.
Published: (2024)

Dropping Experts, Recombining Neurons: Retraining-Free Pruning for Sparse Mixture-of-Experts LLMs
by: Zhou, Yixiao, et al.
Published: (2025)

DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
by: Dai, Damai, et al.
Published: (2024)

DiEP: Adaptive Mixture-of-Experts Compression through Differentiable Expert Pruning
by: Bai, Sikai, et al.
Published: (2025)

Dynamic Experts Search: Enhancing Reasoning in Mixture-of-Experts LLMs at Test Time
by: Han, Yixuan, et al.
Published: (2025)

Native Parallel Reasoner: Reasoning in Parallelism via Self-Distilled Reinforcement Learning
by: Wu, Tong, et al.
Published: (2025)

Chain-of-Experts: Unlocking the Communication Power of Mixture-of-Experts Models
by: Wang, Zihan, et al.
Published: (2025)

Sparser is Faster and Less is More: Efficient Sparse Attention for Long-Range Transformers
by: Lou, Chao, et al.
Published: (2024)

GatePro: Parameter-Free Expert Selection Optimization for Mixture-of-Experts Models
by: Zheng, Chen, et al.
Published: (2025)

LangSuitE: Planning, Controlling and Interacting with Large Language Models in Embodied Text Environments
by: Jia, Zixia, et al.
Published: (2024)

MoBiLE: Efficient Mixture-of-Experts Inference on Consumer GPU with Mixture of Big Little Experts
by: Zhao, Yushu, et al.
Published: (2025)

Mixture-of-Prompt-Experts for Multi-modal Semantic Understanding
by: Wu, Zichen, et al.
Published: (2024)

Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models
by: Qiu, Zihan, et al.
Published: (2025)

Mixture of Neuron Experts
by: Cheng, Runxi, et al.
Published: (2025)

Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM
by: Sukhbaatar, Sainbayar, et al.
Published: (2024)

Rewiring Experts on the Fly:Continuous Rerouting for Better Online Adaptation in Mixture-of-Expert models
by: Su, Guinan, et al.
Published: (2025)

HEBATRON: A Hebrew-Specialized Open-Weight Mixture-of-Experts Language Model
by: Kayzer, Noam, et al.
Published: (2026)

Multi-Type Context-Aware Conversational Recommender Systems via Mixture-of-Experts
by: Zou, Jie, et al.
Published: (2025)

Efficiently Democratizing Medical LLMs for 50 Languages via a Mixture of Language Family Experts
by: Zheng, Guorui, et al.
Published: (2024)

ReXMoE: Reusing Experts with Minimal Overhead in Mixture-of-Experts
by: Tan, Zheyue, et al.
Published: (2025)

Adaptive Preference Optimization with Uncertainty-aware Utility Anchor
by: Wang, Xiaobo, et al.
Published: (2025)

Combining Supervised Learning and Reinforcement Learning for Multi-Label Classification Tasks with Partial Labels
by: Jia, Zixia, et al.
Published: (2024)

Nexus: Specialization meets Adaptability for Efficiently Training Mixture of Experts
by: Gritsch, Nikolas, et al.
Published: (2024)

Unveiling Super Experts in Mixture-of-Experts Large Language Models
by: Su, Zunhai, et al.
Published: (2025)

LongFaith: Enhancing Long-Context Reasoning in LLMs with Faithful Synthetic Data
by: Yang, Cehao, et al.
Published: (2025)

Mixture of Lookup Experts
by: Jie, Shibo, et al.
Published: (2025)

Parameter-Efficient Fine-Tuning of LLMs with Mixture of Space Experts
by: Zhang, Buze, et al.
Published: (2026)

Diversifying the Expert Knowledge for Task-Agnostic Pruning in Sparse Mixture-of-Experts
by: Zhang, Zeliang, et al.
Published: (2024)

LEAD: Layer-wise Expert-aligned Decoding for Faithful Radiology Report Generation
by: Yang, Ruixiao, et al.
Published: (2026)

Dynamic Data Mixing Maximizes Instruction Tuning for Mixture-of-Experts
by: Zhu, Tong, et al.
Published: (2024)