Imagen de Portada

MoE-Gen: High-Throughput MoE Inference on a Single GPU with Module-Based Batching

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Xu, Tairan, Xue, Leyang, Lu, Zhan, Jackson, Adrian, Mai, Luo
Formato:	Preprint
Publicado:	2025
Materias:	Distributed, Parallel, and Cluster Computing Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

MoE-Lightning: High-Throughput MoE Inference on Memory-constrained GPUs
por: Cao, Shiyi, et al.
Publicado: (2024)

HarMoEny: Efficient Multi-GPU Inference of MoE Models
por: Doucet, Zachary, et al.
Publicado: (2025)

Staleness-Centric Optimizations for Parallel Diffusion MoE Inference
por: Luo, Jiajun, et al.
Publicado: (2024)

MoE-CAP: Benchmarking Cost, Accuracy and Performance of Sparse Mixture-of-Experts Systems
por: Jiang, Yinsicheng, et al.
Publicado: (2025)

MoE-CAP: Benchmarking Cost, Accuracy and Performance of Sparse Mixture-of-Experts Systems
por: Jiang, Yinsicheng, et al.
Publicado: (2024)

SP-MoE: Speculative Decoding and Prefetching for Accelerating MoE-based Model Inference
por: Chen, Liangkun, et al.
Publicado: (2025)

OD-MoE: On-Demand Expert Loading for Cacheless Edge-Distributed MoE Inference
por: Wang, Liujianfu, et al.
Publicado: (2025)

GRACE-MoE: Grouping and Replication with Locality-Aware Routing for Efficient Distributed MoE Inference
por: Han, Yu, et al.
Publicado: (2025)

MoE-Lens: Towards the Hardware Limit of High-Throughput MoE LLM Serving Under Resource Constraints
por: Yuan, Yichao, et al.
Publicado: (2025)

Accelerating Distributed MoE Training and Inference with Lina
por: Li, Jiamin, et al.
Publicado: (2022)

EPS-MoE: Expert Pipeline Scheduler for Cost-Efficient MoE Inference
por: Qian, Yulei, et al.
Publicado: (2024)

MoE-Compression: How the Compression Error of Experts Affects the Inference Accuracy of MoE Model?
por: Ma, Songkai, et al.
Publicado: (2025)

LSH-MoE: Communication-efficient MoE Training via Locality-Sensitive Hashing
por: Nie, Xiaonan, et al.
Publicado: (2024)

Efficient CPU-GPU Collaborative Inference for MoE-based LLMs on Memory-Limited Systems
por: Huang, En-Ming, et al.
Publicado: (2025)

HybriMoE: Hybrid CPU-GPU Scheduling and Cache Management for Efficient MoE Inference
por: Zhong, Shuzhang, et al.
Publicado: (2025)

TriMoE: Augmenting GPU with AMX-Enabled CPU and DIMM-NDP for High-Throughput MoE Inference via Offloading
por: Pan, Yudong, et al.
Publicado: (2026)

Janus: Disaggregating Attention and Experts for Scalable MoE Inference
por: Zhang, Zhexiang, et al.
Publicado: (2025)

MoE-Hub: Taming Software Complexity for Seamless MoE Overlap with Hardware-Accelerated Communication on Multi-GPU Systems
por: Zhou, Zhuoshan, et al.
Publicado: (2026)

Multi-Layer Scheduling for MoE-Based LLM Reasoning
por: Sun, Yifan, et al.
Publicado: (2026)

A Scheduling Framework for Efficient MoE Inference on Edge GPU-NDP Systems
por: Wu, Qi, et al.
Publicado: (2026)

ReviveMoE: Fast Recovery for Hardware Failures in Large-Scale MoE LLM Inference Deployments
por: Li, Haley, et al.
Publicado: (2026)

Lynx: Enabling Efficient MoE Inference through Dynamic Batch-Aware Expert Selection
por: Gupta, Vima, et al.
Publicado: (2024)

Accelerating Edge Inference for Distributed MoE Models with Latency-Optimized Expert Placement
por: Wu, Tian, et al.
Publicado: (2025)

ReaLB: Real-Time Load Balancing for Multimodal MoE Inference
por: Wang, Yingping, et al.
Publicado: (2026)

Surviving Partial Rank Failures in Wide Expert-Parallel MoE Inference
por: Sun, Xun, et al.
Publicado: (2026)

Fine-grained MoE Load Balancing with Linear Programming
por: Zhao, Chenqi, et al.
Publicado: (2025)

Sparse Checkpointing for Fast and Reliable MoE Training
por: Gandhi, Swapnil, et al.
Publicado: (2024)

Hexa-MoE: Efficient and Heterogeneous-aware Training for Mixture-of-Experts
por: Luo, Shuqing, et al.
Publicado: (2024)

BrownoutServe: SLO-Aware Inference Serving under Bursty Workloads for MoE-based LLMs
por: Hu, Jianmin, et al.
Publicado: (2025)

Making MoE-based LLM Inference Resilient with Tarragon
por: Zhang, Songyu, et al.
Publicado: (2026)

eMoE: Task-aware Memory Efficient Mixture-of-Experts-Based (MoE) Model Inference
por: Tairin, Suraiya, et al.
Publicado: (2025)

D$^{2}$MoE: Dual Routing and Dynamic Scheduling for Efficient On-Device MoE-based LLM Serving
por: Wang, Haodong, et al.
Publicado: (2025)

MPipeMoE: Memory Efficient MoE for Pre-trained Models with Adaptive Pipeline Parallelism
por: Zhang, Zheng, et al.
Publicado: (2025)

MemFine: Memory-Aware Fine-Grained Scheduling for MoE Training
por: Zhao, Lu, et al.
Publicado: (2025)

MoE Parallel Folding: Heterogeneous Parallelism Mappings for Efficient Large-Scale MoE Model Training with Megatron Core
por: Liu, Dennis, et al.
Publicado: (2025)

GEM: GPU-Variability-Aware Expert to GPU Mapping for MoE Systems
por: Wawdhane, Sourish, et al.
Publicado: (2026)

FlashMoE: Fast Distributed MoE in a Single Kernel
por: Aimuyo, Osayamen Jonathan, et al.
Publicado: (2025)

Accelerating MoE Model Inference with Expert Sharding
por: Balmau, Oana, et al.
Publicado: (2025)

PROBE: Co-Balancing Computation and Communication in MoE Inference via Real-Time Predictive Prefetching
por: Zhu, Qianchao, et al.
Publicado: (2026)

DuoServe-MoE: Dual-Phase Expert Prefetch and Caching for LLM Inference QoS Assurance
por: Zhang, Yuning, et al.
Publicado: (2025)