Copertina

Polysemantic Experts, Monosemantic Paths: Routing as Control in MoEs

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Ye, Charles, Yuan, Bo, Sharkey, Lee
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Artificial Intelligence Computation and Language Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Continual Pre-training of MoEs: How robust is your router?
di: Thérien, Benjamin, et al.
Pubblicazione: (2025)

Prediction Is All MoE Needs: Expert Load Distribution Goes from Fluctuating to Stabilizing
di: Cong, Peizhuang, et al.
Pubblicazione: (2024)

$μ$-MoE: Test-Time Pruning as Micro-Grained Mixture-of-Experts
di: Koike-Akino, Toshiaki, et al.
Pubblicazione: (2025)

Leave It to the Experts: Detecting Knowledge Distillation via MoE Expert Signatures
di: Li, Pingzhi, et al.
Pubblicazione: (2025)

Dynamic Expert Specialization: Towards Catastrophic Forgetting-Free Multi-Domain MoE Adaptation
di: Li, Junzhuo, et al.
Pubblicazione: (2025)

Elastic MoE: Unlocking the Inference-Time Scalability of Mixture-of-Experts
di: Gu, Naibin, et al.
Pubblicazione: (2025)

Alloc-MoE: Budget-Aware Expert Activation Allocation for Efficient Mixture-of-Experts Inference
di: Liu, Baihui, et al.
Pubblicazione: (2026)

MoE-SpAc: Efficient MoE Inference Based on Speculative Activation Utility in Heterogeneous Edge Scenarios
di: Li, Shuhuai, et al.
Pubblicazione: (2026)

Joint MoE Scaling Laws: Mixture of Experts Can Be Memory Efficient
di: Ludziejewski, Jan, et al.
Pubblicazione: (2025)

MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts
di: Pióro, Maciej, et al.
Pubblicazione: (2024)

MoEs Are Stronger than You Think: Hyper-Parallel Inference Scaling with RoE
di: Zibakhsh, Soheil, et al.
Pubblicazione: (2025)

Post-Trained MoE Can Skip Half Experts via Self-Distillation
di: Lv, Xingtai, et al.
Pubblicazione: (2026)

GRIN: GRadient-INformed MoE
di: Liu, Liyuan, et al.
Pubblicazione: (2024)

SlimQwen: Exploring the Pruning and Distillation in Large MoE Model Pre-training
di: Tang, Shengkun, et al.
Pubblicazione: (2026)

MoE-Pruner: Pruning Mixture-of-Experts Large Language Model using the Hints from Its Router
di: Xie, Yanyue, et al.
Pubblicazione: (2024)

Advancing MoE Efficiency: A Collaboration-Constrained Routing (C2R) Strategy for Better Expert Parallelism Design
di: Zhang, Mohan, et al.
Pubblicazione: (2025)

Deconstructing Pre-training: Knowledge Attribution Analysis in MoE and Dense Models
di: Wang, Bo, et al.
Pubblicazione: (2026)

Capturing Polysemanticity with PRISM: A Multi-Concept Feature Description Framework
di: Kopf, Laura, et al.
Pubblicazione: (2025)

CP-MoE: Consistency-Preserving Mixture-of-Experts for Continual Learning
di: Liu, Yang, et al.
Pubblicazione: (2026)

Linear-MoE: Linear Sequence Modeling Meets Mixture-of-Experts
di: Sun, Weigao, et al.
Pubblicazione: (2025)

Towards an empirical understanding of MoE design choices
di: Fan, Dongyang, et al.
Pubblicazione: (2024)

A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learning
di: Yadav, Prateek, et al.
Pubblicazione: (2024)

MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts
di: Jin, Peng, et al.
Pubblicazione: (2024)

LocMoE: A Low-Overhead MoE for Large Language Model Training
di: Li, Jing, et al.
Pubblicazione: (2024)

Routing-Free Mixture-of-Experts
di: Liu, Yilun, et al.
Pubblicazione: (2026)

Multilingual Routing in Mixture-of-Experts
di: Bandarkar, Lucas, et al.
Pubblicazione: (2025)

MESA: Improving MoE Safety Alignment via Decentralized Expertise
di: Sun, Yitong, et al.
Pubblicazione: (2026)

Turn Waste into Worth: Rectifying Top-$k$ Router of MoE
di: Zeng, Zhiyuan, et al.
Pubblicazione: (2024)

Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routers
di: Ma, Wenhan, et al.
Pubblicazione: (2025)

Mix-MoE: Improving Multilingual Machine Translation of Large Language Models through Mixed MoEs
di: Li, Bo, et al.
Pubblicazione: (2026)

Jakiro: Boosting Speculative Decoding with Decoupled Multi-Head via MoE
di: Huang, Haiduo, et al.
Pubblicazione: (2025)

Yuan3.0 Ultra: A Trillion-Parameter Enterprise-Oriented MoE LLM
di: ai, YuanLab., et al.
Pubblicazione: (2026)

Awakening Dormant Experts:Counterfactual Routing to Mitigate MoE Hallucinations
di: Hu, Wentao, et al.
Pubblicazione: (2026)

Finding Fantastic Experts in MoEs: A Unified Study for Expert Dropping Strategies and Observations
di: Jaiswal, Ajay, et al.
Pubblicazione: (2025)

Mix- and MoE-DPO: A Variational Inference Approach to Direct Preference Optimization
di: Bohne, Jason, et al.
Pubblicazione: (2025)

Breaking the MoE LLM Trilemma: Dynamic Expert Clustering with Structured Compression
di: Zhu, Peijun, et al.
Pubblicazione: (2025)

Probing Semantic Routing in Large Mixture-of-Expert Models
di: Olson, Matthew Lyle, et al.
Pubblicazione: (2025)

MoIN: Mixture of Introvert Experts to Upcycle an LLM
di: Tejankar, Ajinkya, et al.
Pubblicazione: (2024)

Scaling Fine-Grained MoE Beyond 50B Parameters: Empirical Evaluation and Practical Insights
di: Krajewski, Jakub, et al.
Pubblicazione: (2025)

SpectR: Dynamically Composing LM Experts with Spectral Routing
di: Fleshman, William, et al.
Pubblicazione: (2025)