Copertina

Comet: Fine-grained Computation-communication Overlapping for Mixture-of-Experts

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Zhang, Shulai, Zheng, Ningxin, Lin, Haibin, Jiang, Ziheng, Bao, Wenlei, Jiang, Chengquan, Hou, Qi, Cui, Weihao, Zheng, Size, Chang, Li-Wen, Chen, Quan, Liu, Xin
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Distributed, Parallel, and Cluster Computing Artificial Intelligence Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

TileLink: Generating Efficient Compute-Communication Overlapping Kernels using Tile-Centric Primitives
di: Zheng, Size, et al.
Pubblicazione: (2025)

FLUX: Fast Software-based Communication Overlap On GPUs Through Kernel Fusion
di: Chang, Li-Wen, et al.
Pubblicazione: (2024)

Triton-distributed: Programming Overlapping Kernels on Distributed AI Systems with the Triton Compiler
di: Zheng, Size, et al.
Pubblicazione: (2025)

MegaScale-MoE: Large-Scale Communication-Efficient Training of Mixture-of-Experts Models in Production
di: Jin, Chao, et al.
Pubblicazione: (2025)

SwiftSpec: Ultra-Low Latency LLM Decoding by Scaling Asynchronous Speculative Decoding
di: Zhang, Ziyi, et al.
Pubblicazione: (2025)

MuxTune: Efficient Multi-Task LLM Fine-Tuning in Multi-Tenant Datacenters via Spatial-Temporal Backbone Multiplexing
di: Xue, Chunyu, et al.
Pubblicazione: (2026)

Lancet: Accelerating Mixture-of-Experts Training via Whole Graph Computation-Communication Overlapping
di: Jiang, Chenyu, et al.
Pubblicazione: (2024)

Towards Fast Setup and High Throughput of GPU Serverless Computing
di: Zhao, Han, et al.
Pubblicazione: (2024)

UniEP: Unified Expert-Parallel MoE MegaKernel for LLM Training
di: Zheng, Size, et al.
Pubblicazione: (2026)

Optimizing Long-context LLM Serving via Fine-grained Sequence Parallelism
di: Li, Cong, et al.
Pubblicazione: (2025)

Harli: SLO-Aware Co-location of LLM Inference and PEFT-based Finetuning on Model-as-a-Service Platforms
di: Xu, Ao, et al.
Pubblicazione: (2025)

MegaScale-Infer: Serving Mixture-of-Experts at Scale with Disaggregated Expert Parallelism
di: Zhu, Ruidong, et al.
Pubblicazione: (2025)

Stable-MoE: Lyapunov-based Token Routing for Distributed Mixture-of-Experts Training over Edge Networks
di: Shi, Long, et al.
Pubblicazione: (2025)

Elastic Mixture of Rank-Wise Experts for Knowledge Reuse in Federated Fine-Tuning
di: Wu, Yebo, et al.
Pubblicazione: (2025)

FedMoE-DA: Federated Mixture of Experts via Domain Aware Fine-grained Aggregation
di: Zhan, Ziwei, et al.
Pubblicazione: (2024)

HAP: Hybrid Adaptive Parallelism for Efficient Mixture-of-Experts Inference
di: Lin, Haoran, et al.
Pubblicazione: (2025)

Accelerating Mixture-of-Experts Inference by Hiding Offloading Latency with Speculative Decoding
di: Wang, Zhibin, et al.
Pubblicazione: (2025)

DisagMoE: Computation-Communication overlapped MoE Training via Disaggregated AF-Pipe Parallelism
di: Zeng, Zhichen, et al.
Pubblicazione: (2026)

MegaScale-Omni: A Hyper-Scale, Workload-Resilient System for MultiModal LLM Training in Production
di: Xue, Chunyu, et al.
Pubblicazione: (2026)

Shortcut-connected Expert Parallelism for Accelerating Mixture-of-Experts
di: Cai, Weilin, et al.
Pubblicazione: (2024)

CCRSat: A Collaborative Computation Reuse Framework for Satellite Edge Computing Networks
di: Zhang, Ye, et al.
Pubblicazione: (2025)

CRAFT: Fine-Grained Cost-Aware Expert Replication For Efficient Mixture-of-Experts Serving
di: Zhao, Adrian, et al.
Pubblicazione: (2026)

Toward Cost-Efficient Serving of Mixture-of-Experts with Asynchrony
di: Wang, Shaoyu, et al.
Pubblicazione: (2025)

ExpertWeave: Efficiently Serving Expert-Specialized Fine-Tuned Adapters at Scale
di: Shi, Ge, et al.
Pubblicazione: (2025)

Lagom: Unleashing the Power of Communication and Computation Overlapping for Distributed LLM Training
di: Xu, Guanbin, et al.
Pubblicazione: (2026)

Cross-region Model Training with Communication-Computation Overlapping and Delay Compensation
di: Zhu, Ying, et al.
Pubblicazione: (2025)

Hexa-MoE: Efficient and Heterogeneous-aware Training for Mixture-of-Experts
di: Luo, Shuqing, et al.
Pubblicazione: (2024)

Towards Fine-Grained Scalability for Stateful Stream Processing Systems
di: Qing, Yunfan, et al.
Pubblicazione: (2025)

Arena: Efficiently Training Large Models via Dynamic Scheduling and Adaptive Parallelism Co-Design
di: Xue, Chunyu, et al.
Pubblicazione: (2024)

Accelerating Sparse DNNs Based on Tiled GEMM
di: Guo, Cong, et al.
Pubblicazione: (2024)

ElasticMoE: An Efficient Auto Scaling Method for Mixture-of-Experts Models
di: Singh, Gursimran, et al.
Pubblicazione: (2025)

Fine-grained MoE Load Balancing with Linear Programming
di: Zhao, Chenqi, et al.
Pubblicazione: (2025)

Exploring Fine-grained Task Parallelism on Simultaneous Multithreading Cores
di: Los, Denis, et al.
Pubblicazione: (2024)

SiDA-MoE: Sparsity-Inspired Data-Aware Serving for Efficient and Scalable Large Mixture-of-Experts Models
di: Du, Zhixu, et al.
Pubblicazione: (2023)

Optimizing Distributed Deployment of Mixture-of-Experts Model Inference in Serverless Computing
di: Liu, Mengfan, et al.
Pubblicazione: (2025)

LAER-MoE: Load-Adaptive Expert Re-layout for Efficient Mixture-of-Experts Training
di: Liu, Xinyi, et al.
Pubblicazione: (2026)

Characterizing Compute-Communication Overlap in GPU-Accelerated Distributed Deep Learning: Performance and Power Implications
di: Lee, Seonho, et al.
Pubblicazione: (2025)

FLASH Viterbi: Fast and Adaptive Viterbi Decoding for Modern Data Systems
di: Deng, Ziheng, et al.
Pubblicazione: (2025)

Collaborative Satellite Computing through Adaptive DNN Task Splitting and Offloading
di: Peng, Shifeng, et al.
Pubblicazione: (2024)

Syncopate: Efficient Multi-GPU AI Kernels via Automatic Chunk-Centric Compute-Communication Overlap
di: Qiang, Xinwei, et al.
Pubblicazione: (2026)