Copertina

MoC-System: Efficient Fault Tolerance for Sparse Mixture-of-Experts Model Training

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Cai, Weilin, Qin, Le, Huang, Jiayi
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Distributed, Parallel, and Cluster Computing Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

DualSparse-MoE: Coordinating Tensor/Neuron-Level Sparsity with Expert Partition and Reconstruction
di: Cai, Weilin, et al.
Pubblicazione: (2025)

Shortcut-connected Expert Parallelism for Accelerating Mixture-of-Experts
di: Cai, Weilin, et al.
Pubblicazione: (2024)

HeterMoE: Efficient Training of Mixture-of-Experts Models on Heterogeneous GPUs
di: Wu, Yongji, et al.
Pubblicazione: (2025)

MoE-CAP: Benchmarking Cost, Accuracy and Performance of Sparse Mixture-of-Experts Systems
di: Jiang, Yinsicheng, et al.
Pubblicazione: (2024)

MoE-CAP: Benchmarking Cost, Accuracy and Performance of Sparse Mixture-of-Experts Systems
di: Jiang, Yinsicheng, et al.
Pubblicazione: (2025)

LAER-MoE: Load-Adaptive Expert Re-layout for Efficient Mixture-of-Experts Training
di: Liu, Xinyi, et al.
Pubblicazione: (2026)

MegaScale-MoE: Large-Scale Communication-Efficient Training of Mixture-of-Experts Models in Production
di: Jin, Chao, et al.
Pubblicazione: (2025)

eMoE: Task-aware Memory Efficient Mixture-of-Experts-Based (MoE) Model Inference
di: Tairin, Suraiya, et al.
Pubblicazione: (2025)

SYMI: Efficient Mixture-of-Experts Training via Model and Optimizer State Decoupling
di: Skiadopoulos, Athinagoras, et al.
Pubblicazione: (2025)

The Streaming Batch Model for Efficient and Fault-Tolerant Heterogeneous Execution
di: Luan, Frank Sifei, et al.
Pubblicazione: (2025)

Lazarus: Resilient and Elastic Training of Mixture-of-Experts Models
di: Wu, Yongji, et al.
Pubblicazione: (2024)

CRAFT: Fine-Grained Cost-Aware Expert Replication For Efficient Mixture-of-Experts Serving
di: Zhao, Adrian, et al.
Pubblicazione: (2026)

MoETuner: Optimized Mixture of Expert Serving with Balanced Expert Placement and Token Routing
di: Go, Seokjin, et al.
Pubblicazione: (2025)

SiDA-MoE: Sparsity-Inspired Data-Aware Serving for Efficient and Scalable Large Mixture-of-Experts Models
di: Du, Zhixu, et al.
Pubblicazione: (2023)

MoE-SpeQ: Speculative Quantized Decoding with Proactive Expert Prefetching and Offloading for Mixture-of-Experts
di: Wang, Wenfeng, et al.
Pubblicazione: (2025)

pFedMoE: Data-Level Personalization with Mixture of Experts for Model-Heterogeneous Personalized Federated Learning
di: Yi, Liping, et al.
Pubblicazione: (2024)

HierMoE: Accelerating MoE Training with Hierarchical Token Deduplication and Expert Swap
di: Lin, Wenxiang, et al.
Pubblicazione: (2025)

X-MoE: Enabling Scalable Training for Emerging Mixture-of-Experts Architectures on HPC Platforms
di: Yuan, Yueming, et al.
Pubblicazione: (2025)

Scalable Training of Mixture-of-Experts Models with Megatron Core
di: Yan, Zijie, et al.
Pubblicazione: (2026)

FedMoE-DA: Federated Mixture of Experts via Domain Aware Fine-grained Aggregation
di: Zhan, Ziwei, et al.
Pubblicazione: (2024)

Role-Based Fault Tolerance System for LLM RL Post-Training
di: Chen, Zhenqian, et al.
Pubblicazione: (2025)

Hexa-MoE: Efficient and Heterogeneous-aware Training for Mixture-of-Experts
di: Luo, Shuqing, et al.
Pubblicazione: (2024)

Scattered Mixture-of-Experts Implementation
di: Tan, Shawn, et al.
Pubblicazione: (2024)

Lancet: Accelerating Mixture-of-Experts Training via Whole Graph Computation-Communication Overlapping
di: Jiang, Chenyu, et al.
Pubblicazione: (2024)

Parm: Efficient Training of Large Sparsely-Activated Models with Dedicated Schedules
di: Pan, Xinglin, et al.
Pubblicazione: (2024)

QoS-Efficient Serving of Multiple Mixture-of-Expert LLMs Using Partial Runtime Reconfiguration
di: Imani, HamidReza, et al.
Pubblicazione: (2025)

Occult: Optimizing Collaborative Communication across Experts for Accelerated Parallel MoE Training and Inference
di: Luo, Shuqing, et al.
Pubblicazione: (2025)

FT K-means: A High-Performance K-means on GPU with Fault Tolerance
di: Wu, Shixun, et al.
Pubblicazione: (2024)

Optimizing Distributed Deployment of Mixture-of-Experts Model Inference in Serverless Computing
di: Liu, Mengfan, et al.
Pubblicazione: (2025)

MegaScale-Infer: Serving Mixture-of-Experts at Scale with Disaggregated Expert Parallelism
di: Zhu, Ruidong, et al.
Pubblicazione: (2025)

MoE Parallel Folding: Heterogeneous Parallelism Mappings for Efficient Large-Scale MoE Model Training with Megatron Core
di: Liu, Dennis, et al.
Pubblicazione: (2025)

MoE-Compression: How the Compression Error of Experts Affects the Inference Accuracy of MoE Model?
di: Ma, Songkai, et al.
Pubblicazione: (2025)

Lynx: Enabling Efficient MoE Inference through Dynamic Batch-Aware Expert Selection
di: Gupta, Vima, et al.
Pubblicazione: (2024)

HOBBIT: A Mixed Precision Expert Offloading System for Fast MoE Inference
di: Tang, Peng, et al.
Pubblicazione: (2024)

Go With The Flow: Churn-Tolerant Decentralized Training of Large Language Models
di: Blagoev, Nikolay, et al.
Pubblicazione: (2025)

SparDL: Distributed Deep Learning Training with Efficient Sparse Communication
di: Zhao, Minjun, et al.
Pubblicazione: (2023)

ElasticMoE: An Efficient Auto Scaling Method for Mixture-of-Experts Models
di: Singh, Gursimran, et al.
Pubblicazione: (2025)

HybriMoE: Hybrid CPU-GPU Scheduling and Cache Management for Efficient MoE Inference
di: Zhong, Shuzhang, et al.
Pubblicazione: (2025)

Linear-MoE: Linear Sequence Modeling Meets Mixture-of-Experts
di: Sun, Weigao, et al.
Pubblicazione: (2025)

FaaSMoE: A Serverless Framework for Multi-Tenant Mixture-of-Experts Serving
di: Wang, Minghe, et al.
Pubblicazione: (2026)