Imagen de Portada

Do Domain-specific Experts exist in MoE-based LLMs?

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Do, Giang, Le, Hung, Tran, Truyen
Formato:	Preprint
Publicado:	2026
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

S2MoE: Robust Sparse Mixture of Experts via Stochastic Learning
por: Do, Giang, et al.
Publicado: (2025)

Rethinking Sparse Mixture of Experts from a Unified Perspective
por: Do, Giang, et al.
Publicado: (2025)

SimSMoE: Solving Representational Collapse via Similarity Measure
por: Do, Giang, et al.
Publicado: (2024)

Eigenvectors of Experts are Training-free Non-collapsing Routers
por: Do, Giang, et al.
Publicado: (2026)

Grove MoE: Towards Efficient and Superior MoE LLMs with Adjugate Experts
por: Wu, Haoyuan, et al.
Publicado: (2025)

On the Role of Discrete Representation in Sparse Mixture of Experts
por: Do, Giang, et al.
Publicado: (2024)

Enhancing Length Extrapolation in Sequential Models with Pointer-Augmented Neural Memory
por: Le, Hung, et al.
Publicado: (2024)

Steering MoE LLMs via Expert (De)Activation
por: Fayyaz, Mohsen, et al.
Publicado: (2025)

EPS-MoE: Expert Pipeline Scheduler for Cost-Efficient MoE Inference
por: Qian, Yulei, et al.
Publicado: (2024)

What Gets Activated: Uncovering Domain and Driver Experts in MoE Language Models
por: Hu, Guimin, et al.
Publicado: (2026)

GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMs
por: Deng, Jianing, et al.
Publicado: (2026)

MH-MoE: Multi-Head Mixture-of-Experts
por: Huang, Shaohan, et al.
Publicado: (2024)

Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts
por: Li, Yunxin, et al.
Publicado: (2024)

$\infty$-MoE: Generalizing Mixture of Experts to Infinite Experts
por: Takashiro, Shota, et al.
Publicado: (2026)

ExpertWeaver: Unlocking the Inherent MoE in Dense LLMs with GLU Activation Patterns
por: Zhao, Ziyu, et al.
Publicado: (2026)

Ada-K Routing: Boosting the Efficiency of MoE-based LLMs
por: Yue, Tongtian, et al.
Publicado: (2024)

Med-MoE: Mixture of Domain-Specific Experts for Lightweight Medical Vision-Language Models
por: Jiang, Songtao, et al.
Publicado: (2024)

OmniMoE: An Efficient MoE by Orchestrating Atomic Experts at Scale
por: Shi, Jingze, et al.
Publicado: (2026)

MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs
por: Xia, Xinfeng, et al.
Publicado: (2025)

Advancing Expert Specialization for Better MoE
por: Guo, Hongcan, et al.
Publicado: (2025)

Pangu Pro MoE: Mixture of Grouped Experts for Efficient Sparsity
por: Tang, Yehui, et al.
Publicado: (2025)

SEER-MoE: Sparse Expert Efficiency through Regularization for Mixture-of-Experts
por: Muzio, Alexandre, et al.
Publicado: (2024)

ROMER: Expert Replacement and Router Calibration for Robust MoE LLMs on Analog Compute-in-Memory Systems
por: Zhou, Wenyong, et al.
Publicado: (2026)

Dynamic Expert Specialization: Towards Catastrophic Forgetting-Free Multi-Domain MoE Adaptation
por: Li, Junzhuo, et al.
Publicado: (2025)

Leave It to the Experts: Detecting Knowledge Distillation via MoE Expert Signatures
por: Li, Pingzhi, et al.
Publicado: (2025)

Expert Selections In MoE Models Reveal (Almost) As Much As Text
por: Nuriyev, Amir, et al.
Publicado: (2026)

PT-MoE: An Efficient Finetuning Framework for Integrating Mixture-of-Experts into Prompt Tuning
por: Li, Zongqian, et al.
Publicado: (2025)

SlimMoE: Structured Compression of Large MoE Models via Expert Slimming and Distillation
por: Li, Zichong, et al.
Publicado: (2025)

MoRAL: MoE Augmented LoRA for LLMs' Lifelong Learning
por: Yang, Shu, et al.
Publicado: (2024)

Pangu Ultra MoE: How to Train Your Big MoE on Ascend NPUs
por: Tang, Yehui, et al.
Publicado: (2025)

BLR-MoE: Boosted Language-Routing Mixture of Experts for Domain-Robust Multilingual E2E ASR
por: Ma, Guodong, et al.
Publicado: (2025)

Harder Tasks Need More Experts: Dynamic Routing in MoE Models
por: Huang, Quzhe, et al.
Publicado: (2024)

Evaluating Expert Contributions in a MoE LLM for Quiz-Based Tasks
por: Chernov, Andrei
Publicado: (2025)

Progressive Multi-granular Alignments for Grounded Reasoning in Large Vision-Language Models
por: Le, Quang-Hung, et al.
Publicado: (2024)

Elastic MoE: Unlocking the Inference-Time Scalability of Mixture-of-Experts
por: Gu, Naibin, et al.
Publicado: (2025)

EAQuant: Enhancing Post-Training Quantization for MoE Models via Expert-Aware Optimization
por: Fu, Zhongqian, et al.
Publicado: (2025)

Expert-Token Resonance MoE: Bidirectional Routing with Efficiency Affinity-Driven Active Selection
por: Li, Jing, et al.
Publicado: (2024)

LLaDA-MoE: A Sparse MoE Diffusion Language Model
por: Zhu, Fengqi, et al.
Publicado: (2025)

Alloc-MoE: Budget-Aware Expert Activation Allocation for Efficient Mixture-of-Experts Inference
por: Liu, Baihui, et al.
Publicado: (2026)

CP-MoE: Consistency-Preserving Mixture-of-Experts for Continual Learning
por: Liu, Yang, et al.
Publicado: (2026)