Image de couverture de livre

Understanding Multilingualism in Mixture-of-Experts LLMs: Routing Mechanism, Expert Specialization, and Layerwise Steering

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Chen, Yuxin, Cai, Zhengzhou, Ji, Xiangtian, Zhao, Weixiang, Zhang, An, Wang, Xiang, Chua, Tat-Seng
Format:	Preprint
Publié:	2026
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Layerwise Recurrent Router for Mixture-of-Experts
par: Qiu, Zihan, et autres
Publié: (2024)

Multilingual Routing in Mixture-of-Experts
par: Bandarkar, Lucas, et autres
Publié: (2025)

Understanding and Leveraging the Expert Specialization of Context Faithfulness in Mixture-of-Experts LLMs
par: Bai, Jun, et autres
Publié: (2025)

Do LLMs and VLMs Share Neurons for Inference? Evidence and Mechanisms of Cross-Modal Transfer
par: Cui, Chenhang, et autres
Publié: (2026)

ExpertSteer: Intervening in LLMs through Expert Knowledge
par: Wang, Weixuan, et autres
Publié: (2025)

Learning to Self-Verify Makes Language Models Better Reasoners
par: Chen, Yuxin, et autres
Publié: (2026)

Routing-Aligned Fine-Tuning for Multilingual Downstream Tasks in Mixture-of-Experts Models
par: Deng, Guanzhi, et autres
Publié: (2026)

Less, but Better: Efficient Multilingual Expansion for LLMs via Layer-wise Mixture-of-Experts
par: Zhang, Xue, et autres
Publié: (2025)

When Are Experts Misrouted? Counterfactual Routing Analysis in Mixture-of-Experts Language Models
par: Yoon, Youngsik, et autres
Publié: (2026)

When Less Language is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasoners
par: Zhao, Weixiang, et autres
Publié: (2025)

Maximum Score Routing For Mixture-of-Experts
par: Dong, Bowen, et autres
Publié: (2025)

Routing by Analogy: kNN-Augmented Expert Assignment for Mixture-of-Experts
par: Lyu, Boxuan, et autres
Publié: (2026)

Reinforced Strategy Optimization for Conversational Recommender Systems via Network-of-Experts
par: Zhao, Xiaoyan, et autres
Publié: (2025)

Routing-Free Mixture-of-Experts
par: Liu, Yilun, et autres
Publié: (2026)

SteerX: Disentangled Steering for LLM Personalization
par: Zhao, Xiaoyan, et autres
Publié: (2025)

MPO: Multilingual Safety Alignment via Reward Gap Optimization
par: Zhao, Weixiang, et autres
Publié: (2025)

AdaSteer: Your Aligned LLM is Inherently an Adaptive Jailbreak Defender
par: Zhao, Weixiang, et autres
Publié: (2025)

MoE-LPR: Multilingual Extension of Large Language Models through Mixture-of-Experts with Language Priors Routing
par: Zhou, Hao, et autres
Publié: (2024)

DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
par: Dai, Damai, et autres
Publié: (2024)

SEUF: Is Unlearning One Expert Enough for Mixture-of-Experts LLMs?
par: Zhuang, Haomin, et autres
Publié: (2024)

Personalized Text Generation with Contrastive Activation Steering
par: Zhang, Jinghao, et autres
Publié: (2025)

Scaling Laws for Upcycling Mixture-of-Experts Language Models
par: Liew, Seng Pei, et autres
Publié: (2025)

An Expert is Worth One Token: Synergizing Multiple Expert LLMs as Generalist via Expert Token Routing
par: Chai, Ziwei, et autres
Publié: (2024)

Understanding the Mixture-of-Experts with Nadaraya-Watson Kernel
par: Zheng, Chuanyang, et autres
Publié: (2025)

Mixture of Experts for Low-Resource LLMs
par: Joseph, Ori Bar, et autres
Publié: (2026)

Mixture Compressor for Mixture-of-Experts LLMs Gains More
par: Huang, Wei, et autres
Publié: (2024)

Dropping Experts, Recombining Neurons: Retraining-Free Pruning for Sparse Mixture-of-Experts LLMs
par: Zhou, Yixiao, et autres
Publié: (2025)

Group then Scale: Dynamic Mixture-of-Experts Multilingual Language Model
par: Li, Chong, et autres
Publié: (2025)

ExpertFlow: Efficient Mixture-of-Experts Inference via Predictive Expert Caching and Token Scheduling
par: He, Xin, et autres
Publié: (2024)

Beyond Benchmarks: Understanding Mixture-of-Experts Models through Internal Mechanisms
par: Ying, Jiahao, et autres
Publié: (2025)

Mixture-of-Supernets: Improving Weight-Sharing Supernet Training with Architecture-Routed Mixture-of-Experts
par: Jawahar, Ganesh, et autres
Publié: (2023)

LD-MoLE: Learnable Dynamic Routing for Mixture of LoRA Experts
par: Zhuang, Yuan, et autres
Publié: (2025)

Seeing but Not Thinking: Routing Distraction in Multimodal Mixture-of-Experts
par: Xu, Haolei, et autres
Publié: (2026)

Mixture of In-Context Experts Enhance LLMs' Long Context Awareness
par: Lin, Hongzhan, et autres
Publié: (2024)

ProtT3: Protein-to-Text Generation for Text-based Protein Understanding
par: Liu, Zhiyuan, et autres
Publié: (2024)

AlphaSteer: Learning Refusal Steering with Principled Null-Space Constraint
par: Sheng, Leheng, et autres
Publié: (2025)

Probing Semantic Routing in Large Mixture-of-Expert Models
par: Olson, Matthew Lyle, et autres
Publié: (2025)

Dynamic Multi-Expert Projectors with Stabilized Routing for Multilingual Speech Recognition
par: Pandey, Isha, et autres
Publié: (2026)

Nexus: Specialization meets Adaptability for Efficiently Training Mixture of Experts
par: Gritsch, Nikolas, et autres
Publié: (2024)

DynMoLE: Boosting Mixture of LoRA Experts Fine-Tuning with a Hybrid Routing Mechanism
par: Li, Dengchun, et autres
Publié: (2025)