Imagen de Portada

Capacity-Aware Inference: Mitigating the Straggler Effect in Mixture of Experts

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	He, Shwai, Cai, Weilin, Huang, Jiayi, Li, Ang
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning Artificial Intelligence Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

What Matters in Transformers? Not All Attention is Needed
por: He, Shwai, et al.
Publicado: (2024)

Towards Efficient Mixture of Experts: A Holistic Study of Compression Techniques
por: He, Shwai, et al.
Publicado: (2024)

Alloc-MoE: Budget-Aware Expert Activation Allocation for Efficient Mixture-of-Experts Inference
por: Liu, Baihui, et al.
Publicado: (2026)

A Unified Virtual Mixture-of-Experts Framework:Enhanced Inference and Hallucination Mitigation in Single-Model System
por: Liu, Mingyan
Publicado: (2025)

MoESD: Mixture of Experts Stable Diffusion to Mitigate Gender Bias
por: Wang, Guorun, et al.
Publicado: (2024)

Multi-Head Mixture-of-Experts
por: Wu, Xun, et al.
Publicado: (2024)

Elastic MoE: Unlocking the Inference-Time Scalability of Mixture-of-Experts
por: Gu, Naibin, et al.
Publicado: (2025)

DualSparse-MoE: Coordinating Tensor/Neuron-Level Sparsity with Expert Partition and Reconstruction
por: Cai, Weilin, et al.
Publicado: (2025)

Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
por: Lu, Xudong, et al.
Publicado: (2024)

Dense Training, Sparse Inference: Rethinking Training of Mixture-of-Experts Language Models
por: Pan, Bowen, et al.
Publicado: (2024)

A Survey on Mixture of Experts in Large Language Models
por: Cai, Weilin, et al.
Publicado: (2024)

Upcycling Large Language Models into Mixture of Experts
por: He, Ethan, et al.
Publicado: (2024)

Shortcut-connected Expert Parallelism for Accelerating Mixture-of-Experts
por: Cai, Weilin, et al.
Publicado: (2024)

Multilingual Routing in Mixture-of-Experts
por: Bandarkar, Lucas, et al.
Publicado: (2025)

Routing-Free Mixture-of-Experts
por: Liu, Yilun, et al.
Publicado: (2026)

Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning
por: Li, Ming, et al.
Publicado: (2024)

MoST: Mixing Speech and Text with Modality-Aware Mixture of Experts
por: Lou, Yuxuan, et al.
Publicado: (2026)

MobileMoE: Scaling On-Device Mixture of Experts
por: Chen, Yanbei, et al.
Publicado: (2026)

SEUF: Is Unlearning One Expert Enough for Mixture-of-Experts LLMs?
por: Zhuang, Haomin, et al.
Publicado: (2024)

MoxE: Mixture of xLSTM Experts with Entropy-Aware Routing for Efficient Language Modeling
por: Thiombiano, Abdoul Majid O., et al.
Publicado: (2025)

On the Spatial Structure of Mixture-of-Experts in Transformers
por: Bershatsky, Daniel, et al.
Publicado: (2025)

MEPT: Mixture of Expert Prompt Tuning as a Manifold Mapper
por: Zeng, Runjia, et al.
Publicado: (2025)

Towards a Comprehensive Scaling Law of Mixture-of-Experts
por: Zhao, Guoliang, et al.
Publicado: (2025)

Mixture of Heterogeneous Grouped Experts for Language Modeling
por: Ma, Zhicheng, et al.
Publicado: (2026)

Scaling Laws for Fine-Grained Mixture of Experts
por: Krajewski, Jakub, et al.
Publicado: (2024)

MoIN: Mixture of Introvert Experts to Upcycle an LLM
por: Tejankar, Ajinkya, et al.
Publicado: (2024)

Dynamic Experts Search: Enhancing Reasoning in Mixture-of-Experts LLMs at Test Time
por: Han, Yixuan, et al.
Publicado: (2025)

Parameter-Efficient Routed Fine-Tuning: Mixture-of-Experts Demands Mixture of Adaptation Modules
por: Liu, Yilun, et al.
Publicado: (2025)

Every Expert Matters: Towards Effective Knowledge Distillation for Mixture-of-Experts Language Models
por: Kim, Gyeongman, et al.
Publicado: (2025)

Mixtures of SubExperts for Large Language Continual Learning
por: Kang, Haeyong
Publicado: (2025)

Probing Semantic Routing in Large Mixture-of-Expert Models
por: Olson, Matthew Lyle, et al.
Publicado: (2025)

Pruning and Distilling Mixture-of-Experts into Dense Language Models
por: Kim, Junhyuck, et al.
Publicado: (2026)

OLMoE: Open Mixture-of-Experts Language Models
por: Muennighoff, Niklas, et al.
Publicado: (2024)

QuantMoE-Bench: Examining Post-Training Quantization for Mixture-of-Experts
por: Li, Pingzhi, et al.
Publicado: (2024)

Reformatted Alignment
por: Fan, Run-Ze, et al.
Publicado: (2024)

Optimal Sparsity of Mixture-of-Experts Language Models for Reasoning Tasks
por: Nakamura, Taishi, et al.
Publicado: (2025)

Nexus: Specialization meets Adaptability for Efficiently Training Mixture of Experts
por: Gritsch, Nikolas, et al.
Publicado: (2024)

Contrastive Learning and Mixture of Experts Enables Precise Vector Embeddings
por: Hallee, Logan, et al.
Publicado: (2024)

Autonomy-of-Experts Models
por: Lv, Ang, et al.
Publicado: (2025)

LeMoLE: LLM-Enhanced Mixture of Linear Experts for Time Series Forecasting
por: Zhang, Lingzheng, et al.
Publicado: (2024)