Image de couverture de livre

MoEBlaze: Breaking the Memory Wall for Efficient MoE Training on Modern GPUs

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zhang, Jiyuan, Liu, Yining, Yan, Siqi, Deng, Lisen, Cao, Jennifer, Yang, Shuqi, Ni, Min, Xue, Bi, Li, Shen
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Artificial Intelligence Distributed, Parallel, and Cluster Computing
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

MoE-Lightning: High-Throughput MoE Inference on Memory-constrained GPUs
par: Cao, Shiyi, et autres
Publié: (2024)

LSH-MoE: Communication-efficient MoE Training via Locality-Sensitive Hashing
par: Nie, Xiaonan, et autres
Publié: (2024)

MemFine: Memory-Aware Fine-Grained Scheduling for MoE Training
par: Zhao, Lu, et autres
Publié: (2025)

Accelerating MoE with Dynamic In-Switch Computing on Multi-GPUs
par: Zhang, Qijun, et autres
Publié: (2026)

Accelerating Distributed MoE Training and Inference with Lina
par: Li, Jiamin, et autres
Publié: (2022)

Sparse Checkpointing for Fast and Reliable MoE Training
par: Gandhi, Swapnil, et autres
Publié: (2024)

MPipeMoE: Memory Efficient MoE for Pre-trained Models with Adaptive Pipeline Parallelism
par: Zhang, Zheng, et autres
Publié: (2025)

SP-MoE: Speculative Decoding and Prefetching for Accelerating MoE-based Model Inference
par: Chen, Liangkun, et autres
Publié: (2025)

OD-MoE: On-Demand Expert Loading for Cacheless Edge-Distributed MoE Inference
par: Wang, Liujianfu, et autres
Publié: (2025)

GRACE-MoE: Grouping and Replication with Locality-Aware Routing for Efficient Distributed MoE Inference
par: Han, Yu, et autres
Publié: (2025)

Revealing the Challenges of Attention-FFN Disaggregation for Modern MoE Models and Hardware Systems
par: Liu, Guowei, et autres
Publié: (2026)

Hexa-MoE: Efficient and Heterogeneous-aware Training for Mixture-of-Experts
par: Luo, Shuqing, et autres
Publié: (2024)

HarMoEny: Efficient Multi-GPU Inference of MoE Models
par: Doucet, Zachary, et autres
Publié: (2025)

MoE Parallel Folding: Heterogeneous Parallelism Mappings for Efficient Large-Scale MoE Model Training with Megatron Core
par: Liu, Dennis, et autres
Publié: (2025)

MoE-Gen: High-Throughput MoE Inference on a Single GPU with Module-Based Batching
par: Xu, Tairan, et autres
Publié: (2025)

UniEP: Unified Expert-Parallel MoE MegaKernel for LLM Training
par: Zheng, Size, et autres
Publié: (2026)

Efficient CPU-GPU Collaborative Inference for MoE-based LLMs on Memory-Limited Systems
par: Huang, En-Ming, et autres
Publié: (2025)

FEPLB: Exploiting Copy Engines for Nearly Free MoE Load Balancing in Distributed Training
par: Qi, Shuyao, et autres
Publié: (2026)

EPS-MoE: Expert Pipeline Scheduler for Cost-Efficient MoE Inference
par: Qian, Yulei, et autres
Publié: (2024)

HierMoE: Accelerating MoE Training with Hierarchical Token Deduplication and Expert Swap
par: Lin, Wenxiang, et autres
Publié: (2025)

Fine-grained MoE Load Balancing with Linear Programming
par: Zhao, Chenqi, et autres
Publié: (2025)

Multi-Layer Scheduling for MoE-Based LLM Reasoning
par: Sun, Yifan, et autres
Publié: (2026)

Staleness-Centric Optimizations for Parallel Diffusion MoE Inference
par: Luo, Jiajun, et autres
Publié: (2024)

Janus: Disaggregating Attention and Experts for Scalable MoE Inference
par: Zhang, Zhexiang, et autres
Publié: (2025)

ReviveMoE: Fast Recovery for Hardware Failures in Large-Scale MoE LLM Inference Deployments
par: Li, Haley, et autres
Publié: (2026)

Surviving Partial Rank Failures in Wide Expert-Parallel MoE Inference
par: Sun, Xun, et autres
Publié: (2026)

eMoE: Task-aware Memory Efficient Mixture-of-Experts-Based (MoE) Model Inference
par: Tairin, Suraiya, et autres
Publié: (2025)

Stable-MoE: Lyapunov-based Token Routing for Distributed Mixture-of-Experts Training over Edge Networks
par: Shi, Long, et autres
Publié: (2025)

MoE-Compression: How the Compression Error of Experts Affects the Inference Accuracy of MoE Model?
par: Ma, Songkai, et autres
Publié: (2025)

GriNNder: Breaking the Memory Capacity Wall in Full-Graph GNN Training with Storage Offloading
par: Song, Jaeyong, et autres
Publié: (2026)

Pro-Prophet: A Systematic Load Balancing Method for Efficient Parallel Training of Large-scale MoE Models
par: Wang, Wei, et autres
Publié: (2024)

HeterMoE: Efficient Training of Mixture-of-Experts Models on Heterogeneous GPUs
par: Wu, Yongji, et autres
Publié: (2025)

Occult: Optimizing Collaborative Communication across Experts for Accelerated Parallel MoE Training and Inference
par: Luo, Shuqing, et autres
Publié: (2025)

ReaLB: Real-Time Load Balancing for Multimodal MoE Inference
par: Wang, Yingping, et autres
Publié: (2026)

D$^{2}$MoE: Dual Routing and Dynamic Scheduling for Efficient On-Device MoE-based LLM Serving
par: Wang, Haodong, et autres
Publié: (2025)

MoE-Lens: Towards the Hardware Limit of High-Throughput MoE LLM Serving Under Resource Constraints
par: Yuan, Yichao, et autres
Publié: (2025)

Efficient MoE Serving in the Memory-Bound Regime: Balance Activated Experts, Not Tokens
par: Yu, Yanpeng, et autres
Publié: (2025)

Accelerating Edge Inference for Distributed MoE Models with Latency-Optimized Expert Placement
par: Wu, Tian, et autres
Publié: (2025)

When MoE Meets Blockchain: A Trustworthy Distributed Framework of Large Models
par: Zhu, Weihao, et autres
Publié: (2025)

MoE-Hub: Taming Software Complexity for Seamless MoE Overlap with Hardware-Accelerated Communication on Multi-GPU Systems
par: Zhou, Zhuoshan, et autres
Publié: (2026)