Copertina

SEER-MoE: Sparse Expert Efficiency through Regularization for Mixture-of-Experts

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Muzio, Alexandre, Sun, Alex, He, Churan
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

$\infty$-MoE: Generalizing Mixture of Experts to Infinite Experts
di: Takashiro, Shota, et al.
Pubblicazione: (2026)

$\texttt{MoE-RBench}$: Towards Building Reliable Language Models with Sparse Mixture-of-Experts
di: Chen, Guanjie, et al.
Pubblicazione: (2024)

Elastic MoE: Unlocking the Inference-Time Scalability of Mixture-of-Experts
di: Gu, Naibin, et al.
Pubblicazione: (2025)

MoE-nD: Per-Layer Mixture-of-Experts Routing for Multi-Axis KV Cache Compression
di: Sun, Libo, et al.
Pubblicazione: (2026)

Alloc-MoE: Budget-Aware Expert Activation Allocation for Efficient Mixture-of-Experts Inference
di: Liu, Baihui, et al.
Pubblicazione: (2026)

Linear-MoE: Linear Sequence Modeling Meets Mixture-of-Experts
di: Sun, Weigao, et al.
Pubblicazione: (2025)

MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts
di: Pióro, Maciej, et al.
Pubblicazione: (2024)

Joint MoE Scaling Laws: Mixture of Experts Can Be Memory Efficient
di: Ludziejewski, Jan, et al.
Pubblicazione: (2025)

$μ$-MoE: Test-Time Pruning as Micro-Grained Mixture-of-Experts
di: Koike-Akino, Toshiaki, et al.
Pubblicazione: (2025)

CP-MoE: Consistency-Preserving Mixture-of-Experts for Continual Learning
di: Liu, Yang, et al.
Pubblicazione: (2026)

FLAME-MoE: A Transparent End-to-End Research Platform for Mixture-of-Experts Language Models
di: Kang, Hao, et al.
Pubblicazione: (2025)

Steering MoE LLMs via Expert (De)Activation
di: Fayyaz, Mohsen, et al.
Pubblicazione: (2025)

Leave It to the Experts: Detecting Knowledge Distillation via MoE Expert Signatures
di: Li, Pingzhi, et al.
Pubblicazione: (2025)

MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs
di: Xia, Xinfeng, et al.
Pubblicazione: (2025)

X-MoE: Enabling Scalable Training for Emerging Mixture-of-Experts Architectures on HPC Platforms
di: Yuan, Yueming, et al.
Pubblicazione: (2025)

Harder Tasks Need More Experts: Dynamic Routing in MoE Models
di: Huang, Quzhe, et al.
Pubblicazione: (2024)

GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMs
di: Deng, Jianing, et al.
Pubblicazione: (2026)

SlimMoE: Structured Compression of Large MoE Models via Expert Slimming and Distillation
di: Li, Zichong, et al.
Pubblicazione: (2025)

Diversifying the Expert Knowledge for Task-Agnostic Pruning in Sparse Mixture-of-Experts
di: Zhang, Zeliang, et al.
Pubblicazione: (2024)

Routers Learn the Geometry of Their Experts: Geometric Coupling in Sparse Mixture-of-Experts
di: Ahrac, Sagi, et al.
Pubblicazione: (2026)

MoE-Pruner: Pruning Mixture-of-Experts Large Language Model using the Hints from Its Router
di: Xie, Yanyue, et al.
Pubblicazione: (2024)

Preserving Long-Tailed Expert Information in Mixture-of-Experts Tuning
di: He, Haoze, et al.
Pubblicazione: (2026)

Self-MoE: Towards Compositional Large Language Models with Self-Specialized Experts
di: Kang, Junmo, et al.
Pubblicazione: (2024)

ExpertWeaver: Unlocking the Inherent MoE in Dense LLMs with GLU Activation Patterns
di: Zhao, Ziyu, et al.
Pubblicazione: (2026)

MoFE: Mixture of Frozen Experts Architecture
di: Seo, Jean, et al.
Pubblicazione: (2025)

Mixture of Tokens: Continuous MoE through Cross-Example Aggregation
di: Antoniak, Szymon, et al.
Pubblicazione: (2023)

MoE-CAP: Benchmarking Cost, Accuracy and Performance of Sparse Mixture-of-Experts Systems
di: Jiang, Yinsicheng, et al.
Pubblicazione: (2024)

MoE-CAP: Benchmarking Cost, Accuracy and Performance of Sparse Mixture-of-Experts Systems
di: Jiang, Yinsicheng, et al.
Pubblicazione: (2025)

MoLEx: Mixture of Layer Experts for Finetuning with Sparse Upcycling
di: Teo, Rachel S. Y., et al.
Pubblicazione: (2025)

Unchosen Experts Can Contribute Too: Unleashing MoE Models' Power by Self-Contrast
di: Shi, Chufan, et al.
Pubblicazione: (2024)

RotMoLE: Enhancing Mixture of Low-Rank Experts through Rotational Gating Mechanism
di: Sun, Mengyang, et al.
Pubblicazione: (2026)

MobileMoE: Scaling On-Device Mixture of Experts
di: Chen, Yanbei, et al.
Pubblicazione: (2026)

Post-Trained MoE Can Skip Half Experts via Self-Distillation
di: Lv, Xingtai, et al.
Pubblicazione: (2026)

Polysemantic Experts, Monosemantic Paths: Routing as Control in MoEs
di: Ye, Charles, et al.
Pubblicazione: (2026)

CartesianMoE: Boosting Knowledge Sharing among Experts via Cartesian Product Routing in Mixture-of-Experts
di: Su, Zhenpeng, et al.
Pubblicazione: (2024)

CoSMoEs: Compact Sparse Mixture of Experts
di: Huber, Patrick, et al.
Pubblicazione: (2025)

MoEMoE: Question Guided Dense and Scalable Sparse Mixture-of-Expert for Multi-source Multi-modal Answering
di: Verma, Vinay Kumar, et al.
Pubblicazione: (2025)

ROMER: Expert Replacement and Router Calibration for Robust MoE LLMs on Analog Compute-in-Memory Systems
di: Zhou, Wenyong, et al.
Pubblicazione: (2026)

CAMERA: Multi-Matrix Joint Compression for MoE Models via Micro-Expert Redundancy Analysis
di: Xu, Yuzhuang, et al.
Pubblicazione: (2025)

MoIN: Mixture of Introvert Experts to Upcycle an LLM
di: Tejankar, Ajinkya, et al.
Pubblicazione: (2024)