Copertina

How Many Experts Are Enough? Towards Optimal Semantic Specialization for Mixture-of-Experts

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Park, Sumin, Park, Noseong
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Are Graph Transformers Necessary? Efficient Long-Range Message Passing with Fractal Nodes in MPNNs
di: Choi, Jeongwhan, et al.
Pubblicazione: (2025)

SEUF: Is Unlearning One Expert Enough for Mixture-of-Experts LLMs?
di: Zhuang, Haomin, et al.
Pubblicazione: (2024)

PANDA: Expanded Width-Aware Message Passing Beyond Rewiring
di: Choi, Jeongwhan, et al.
Pubblicazione: (2024)

Exploring Expert Specialization through Unsupervised Training in Sparse Mixture of Experts
di: Nikolic, Strahinja, et al.
Pubblicazione: (2025)

Toward Structural Multimodal Representations: Specialization, Selection, and Sparsification via Mixture-of-Experts
di: Choi, Hahyeon, et al.
Pubblicazione: (2026)

SPI-GAN: Denoising Diffusion GANs with Straight-Path Interpolations
di: Jeon, Jinsung, et al.
Pubblicazione: (2022)

HINTS: Extraction of Human Insights from Time-Series Without External Sources
di: Jhin, Sheo Yon, et al.
Pubblicazione: (2025)

Speculating Experts Accelerates Inference for Mixture-of-Experts
di: Madan, Vivan, et al.
Pubblicazione: (2026)

Optimal Expert-Attention Allocation in Mixture-of-Experts: A Scalable Law for Dynamic Model Design
di: Li, Junzhuo, et al.
Pubblicazione: (2026)

LatentMoE: Toward Optimal Accuracy per FLOP and Parameter in Mixture of Experts
di: Elango, Venmugil, et al.
Pubblicazione: (2026)

HyperMoE: Towards Better Mixture of Experts via Transferring Among Experts
di: Zhao, Hao, et al.
Pubblicazione: (2024)

Uncovering Intra-expert Activation Sparsity for Efficient Mixture-of-Expert Model Execution
di: Park, Jongseok, et al.
Pubblicazione: (2026)

Scaling Multi-Node Mixture-of-Experts Inference Using Expert Activation Patterns
di: Bambhaniya, Abhimanyu, et al.
Pubblicazione: (2026)

Mixture of Raytraced Experts
di: Perin, Andrea, et al.
Pubblicazione: (2025)

AnyExperts: On-Demand Expert Allocation for Multimodal Language Models with Mixture of Expert
di: Gao, Yuting, et al.
Pubblicazione: (2025)

Efficiently Editing Mixture-of-Experts Models with Compressed Experts
di: He, Yifei, et al.
Pubblicazione: (2025)

MoDEx: Mixture of Depth-specific Experts for Multivariate Long-term Time Series Forecasting
di: Yoon, Hyekyung, et al.
Pubblicazione: (2026)

The Illusion of Specialization: Unveiling the Domain-Invariant "Standing Committee" in Mixture-of-Experts Models
di: Wang, Yan, et al.
Pubblicazione: (2026)

One Prompt is not Enough: Automated Construction of a Mixture-of-Expert Prompts
di: Wang, Ruochen, et al.
Pubblicazione: (2024)

Learning Advanced Self-Attention for Linear Transformers in the Singular Value Domain
di: Wi, Hyowon, et al.
Pubblicazione: (2025)

Graph Signal Processing Meets Mamba2: Adaptive Filter Bank via Delta Modulation
di: Shin, Yehjin, et al.
Pubblicazione: (2026)

Nexus: Specialization meets Adaptability for Efficiently Training Mixture of Experts
di: Gritsch, Nikolas, et al.
Pubblicazione: (2024)

Accelerating Mixture-of-Expert Inference with Adaptive Expert Split Mechanism
di: Yan, Jiaming, et al.
Pubblicazione: (2025)

Expert Upcycling: Shifting the Compute-Efficient Frontier of Mixture-of-Experts
di: Dwivedi, Chaitanya, et al.
Pubblicazione: (2026)

Sparsity and Superposition in Mixture of Experts
di: Chaudhari, Marmik, et al.
Pubblicazione: (2025)

Mixture of Diverse Size Experts
di: Sun, Manxi, et al.
Pubblicazione: (2024)

Mixture of Concept Bottleneck Experts
di: De Santis, Francesco, et al.
Pubblicazione: (2026)

Mixture of A Million Experts
di: He, Xu Owen
Pubblicazione: (2024)

Mixture of Experts in a Mixture of RL settings
di: Willi, Timon, et al.
Pubblicazione: (2024)

Probing Semantic Routing in Large Mixture-of-Expert Models
di: Olson, Matthew Lyle, et al.
Pubblicazione: (2025)

Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference
di: Chu, Kexin, et al.
Pubblicazione: (2025)

Peirce in the Machine: How Mixture of Experts Models Perform Hypothesis Construction
di: Rushing, Bruce
Pubblicazione: (2024)

Modeling Expert Interactions in Sparse Mixture of Experts via Graph Structures
di: Nguyen-Nhat, Minh-Khoi, et al.
Pubblicazione: (2025)

UniPool: A Globally Shared Expert Pool for Mixture-of-Experts
di: Huang, Minbin, et al.
Pubblicazione: (2026)

Understanding Expert Structures on Minimax Parameter Estimation in Contaminated Mixture of Experts
di: Yan, Fanqi, et al.
Pubblicazione: (2024)

MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts
di: Jin, Peng, et al.
Pubblicazione: (2024)

Least-Loaded Expert Parallelism: Load Balancing An Imbalanced Mixture-of-Experts
di: Nguyen, Xuan-Phi, et al.
Pubblicazione: (2026)

Every Expert Matters: Towards Effective Knowledge Distillation for Mixture-of-Experts Language Models
di: Kim, Gyeongman, et al.
Pubblicazione: (2025)

Towards Efficient Mixture of Experts: A Holistic Study of Compression Techniques
di: He, Shwai, et al.
Pubblicazione: (2024)

Towards Generalization-Oriented Models for Vehicle Routing Problems with Mixture-of-Experts
di: Miao, Changhao, et al.
Pubblicazione: (2026)