Copertina

HierMoE: Accelerating MoE Training with Hierarchical Token Deduplication and Expert Swap

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Lin, Wenxiang, Pan, Xinglin, Zhang, Lin, Shi, Shaohuai, Wang, Xuan, Chu, Xiaowen
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Distributed, Parallel, and Cluster Computing Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Efficient MoE Inference with Fine-Grained Scheduling of Disaggregated Expert Parallelism
di: Pan, Xinglin, et al.
Pubblicazione: (2025)

ZipCCL: Efficient Lossless Data Compression of Communication Collectives for Accelerating LLM Training
di: Lin, Wenxiang, et al.
Pubblicazione: (2026)

Parm: Efficient Training of Large Sparsely-Activated Models with Dedicated Schedules
di: Pan, Xinglin, et al.
Pubblicazione: (2024)

Janus: Disaggregating Attention and Experts for Scalable MoE Inference
di: Zhang, Zhexiang, et al.
Pubblicazione: (2025)

OD-MoE: On-Demand Expert Loading for Cacheless Edge-Distributed MoE Inference
di: Wang, Liujianfu, et al.
Pubblicazione: (2025)

Occult: Optimizing Collaborative Communication across Experts for Accelerated Parallel MoE Training and Inference
di: Luo, Shuqing, et al.
Pubblicazione: (2025)

MoE-Compression: How the Compression Error of Experts Affects the Inference Accuracy of MoE Model?
di: Ma, Songkai, et al.
Pubblicazione: (2025)

Accelerating MoE Model Inference with Expert Sharding
di: Balmau, Oana, et al.
Pubblicazione: (2025)

Accelerating Distributed MoE Training and Inference with Lina
di: Li, Jiamin, et al.
Pubblicazione: (2022)

Stable-MoE: Lyapunov-based Token Routing for Distributed Mixture-of-Experts Training over Edge Networks
di: Shi, Long, et al.
Pubblicazione: (2025)

LAER-MoE: Load-Adaptive Expert Re-layout for Efficient Mixture-of-Experts Training
di: Liu, Xinyi, et al.
Pubblicazione: (2026)

SP-MoE: Speculative Decoding and Prefetching for Accelerating MoE-based Model Inference
di: Chen, Liangkun, et al.
Pubblicazione: (2025)

MegaScale-MoE: Large-Scale Communication-Efficient Training of Mixture-of-Experts Models in Production
di: Jin, Chao, et al.
Pubblicazione: (2025)

Hexa-MoE: Efficient and Heterogeneous-aware Training for Mixture-of-Experts
di: Luo, Shuqing, et al.
Pubblicazione: (2024)

LSH-MoE: Communication-efficient MoE Training via Locality-Sensitive Hashing
di: Nie, Xiaonan, et al.
Pubblicazione: (2024)

MoE Parallel Folding: Heterogeneous Parallelism Mappings for Efficient Large-Scale MoE Model Training with Megatron Core
di: Liu, Dennis, et al.
Pubblicazione: (2025)

eMoE: Task-aware Memory Efficient Mixture-of-Experts-Based (MoE) Model Inference
di: Tairin, Suraiya, et al.
Pubblicazione: (2025)

Accelerating Edge Inference for Distributed MoE Models with Latency-Optimized Expert Placement
di: Wu, Tian, et al.
Pubblicazione: (2025)

DreamDDP: Accelerating Data Parallel Distributed LLM Training with Layer-wise Scheduled Partial Synchronization
di: Tang, Zhenheng, et al.
Pubblicazione: (2025)

GRACE-MoE: Grouping and Replication with Locality-Aware Routing for Efficient Distributed MoE Inference
di: Han, Yu, et al.
Pubblicazione: (2025)

EPS-MoE: Expert Pipeline Scheduler for Cost-Efficient MoE Inference
di: Qian, Yulei, et al.
Pubblicazione: (2024)

MoE-Gen: High-Throughput MoE Inference on a Single GPU with Module-Based Batching
di: Xu, Tairan, et al.
Pubblicazione: (2025)

UniEP: Unified Expert-Parallel MoE MegaKernel for LLM Training
di: Zheng, Size, et al.
Pubblicazione: (2026)

MoE-SpeQ: Speculative Quantized Decoding with Proactive Expert Prefetching and Offloading for Mixture-of-Experts
di: Wang, Wenfeng, et al.
Pubblicazione: (2025)

HOBBIT: A Mixed Precision Expert Offloading System for Fast MoE Inference
di: Tang, Peng, et al.
Pubblicazione: (2024)

MoE-CAP: Benchmarking Cost, Accuracy and Performance of Sparse Mixture-of-Experts Systems
di: Jiang, Yinsicheng, et al.
Pubblicazione: (2025)

MoE-CAP: Benchmarking Cost, Accuracy and Performance of Sparse Mixture-of-Experts Systems
di: Jiang, Yinsicheng, et al.
Pubblicazione: (2024)

Sparse Checkpointing for Fast and Reliable MoE Training
di: Gandhi, Swapnil, et al.
Pubblicazione: (2024)

Surviving Partial Rank Failures in Wide Expert-Parallel MoE Inference
di: Sun, Xun, et al.
Pubblicazione: (2026)

From Tokens to Layers: Redefining Stall-Free Scheduling for MoE Serving with Layered Prefill
di: Lee, Gunjun, et al.
Pubblicazione: (2025)

Lynx: Enabling Efficient MoE Inference through Dynamic Batch-Aware Expert Selection
di: Gupta, Vima, et al.
Pubblicazione: (2024)

DualSparse-MoE: Coordinating Tensor/Neuron-Level Sparsity with Expert Partition and Reconstruction
di: Cai, Weilin, et al.
Pubblicazione: (2025)

DisagMoE: Computation-Communication overlapped MoE Training via Disaggregated AF-Pipe Parallelism
di: Zeng, Zhichen, et al.
Pubblicazione: (2026)

MoE-Lightning: High-Throughput MoE Inference on Memory-constrained GPUs
di: Cao, Shiyi, et al.
Pubblicazione: (2024)

HybriMoE: Hybrid CPU-GPU Scheduling and Cache Management for Efficient MoE Inference
di: Zhong, Shuzhang, et al.
Pubblicazione: (2025)

X-MoE: Enabling Scalable Training for Emerging Mixture-of-Experts Architectures on HPC Platforms
di: Yuan, Yueming, et al.
Pubblicazione: (2025)

Efficient MoE Serving in the Memory-Bound Regime: Balance Activated Experts, Not Tokens
di: Yu, Yanpeng, et al.
Pubblicazione: (2025)

MoE-Lens: Towards the Hardware Limit of High-Throughput MoE LLM Serving Under Resource Constraints
di: Yuan, Yichao, et al.
Pubblicazione: (2025)

MoEBlaze: Breaking the Memory Wall for Efficient MoE Training on Modern GPUs
di: Zhang, Jiyuan, et al.
Pubblicazione: (2026)

Making MoE-based LLM Inference Resilient with Tarragon
di: Zhang, Songyu, et al.
Pubblicazione: (2026)