Cover Image

AsyncMesh: Fully Asynchronous Optimization for Data and Pipeline Parallelism

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ajanthan, Thalaiyasingam, Ramasinghe, Sameera, Avraham, Gil, Dolatabadi, Hadi Mohaghegh, Koneputugodage, Chamin P Hewa, Shevchenko, Violetta, Zuo, Yan, Long, Alexander
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Distributed, Parallel, and Cluster Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SENTINEL: Stagewise Integrity Verification for Pipeline Parallel Decentralized Training
by: Dolatabadi, Hadi Mohaghegh, et al.
Published: (2026)

Nesterov Method for Asynchronous Pipeline Parallel Optimization
by: Ajanthan, Thalaiyasingam, et al.
Published: (2025)

NuMuon: Nuclear-Norm-Constrained Muon for Compressible LLM Training
by: Dolatabadi, Hadi Mohaghegh, et al.
Published: (2026)

AsyncHZP: Hierarchical ZeRO Parallelism with Asynchronous Scheduling for Scalable LLM Training
by: Bai, Huawei, et al.
Published: (2025)

AsyncSparse: Accelerating Sparse Matrix-Matrix Multiplication on Asynchronous GPU Architectures
by: Liu, Jie, et al.
Published: (2026)

AsyncSC: An Asynchronous Sidechain for Multi-Domain Data Exchange in Internet of Things
by: Yang, Lingxiao, et al.
Published: (2024)

Balancing Pipeline Parallelism with Vocabulary Parallelism
by: Yeung, Man Tsung, et al.
Published: (2024)

Synergistic Tensor and Pipeline Parallelism
by: Qi, Mengshi, et al.
Published: (2025)

AMDP: Asynchronous Multi-Directional Pipeline Parallelism for Large-Scale Models Training
by: Chen, Ling, et al.
Published: (2026)

Fantasy: Efficient Large-scale Vector Search on GPU Clusters with GPUDirect Async
by: Liu, Yi, et al.
Published: (2025)

Tasking framework for Adaptive Speculative Parallel Mesh Generation
by: Tsolakis, Christos, et al.
Published: (2024)

Distributed Semi-Speculative Parallel Anisotropic Mesh Adaptation
by: Garner, Kevin, et al.
Published: (2026)

Efficient Data-Parallel Continual Learning with Asynchronous Distributed Rehearsal Buffers
by: Bouvier, Thomas, et al.
Published: (2024)

Boosting Blockchain Throughput: Parallel EVM Execution with Asynchronous Storage for Reddio
by: Qi, Xiaodong, et al.
Published: (2025)

Mitigating Staleness in Asynchronous Pipeline Parallelism via Basis Rotation
by: Jung, Hyunji, et al.
Published: (2026)

Adaptra: Straggler-Resilient Hybrid-Parallel Training with Pipeline Adaptation
by: Wu, Tianyuan, et al.
Published: (2025)

APEX: Asynchronous Parallel CPU-GPU Execution for Online LLM Inference on Constrained GPUs
by: Fan, Jiakun, et al.
Published: (2025)

TimelyFreeze: Adaptive Parameter Freezing Mechanism for Pipeline Parallelism
by: Cho, Seonghye, et al.
Published: (2026)

A Flexible Programmable Pipeline Parallelism Framework for Efficient DNN Training
by: Jiang, Lijuan, et al.
Published: (2025)

DawnPiper: A Memory-scablable Pipeline Parallel Training Framework
by: Peng, Xuan, et al.
Published: (2025)

Parallelizing Drug Discovery: HPC Pipelines for Alzheimer's Molecular Docking and Simulation
by: Alliata, Paul Ruiz, et al.
Published: (2025)

An Asynchronous Distributed-Memory Parallel Algorithm for k-mer Counting
by: Hati, Souvadra, et al.
Published: (2025)

Heimdall++: Optimizing GPU Utilization and Pipeline Parallelism for Efficient Single-Pulse Detection
by: Xia, Bingzheng, et al.
Published: (2025)

Accelerating Microswimmer Simulations via a Heterogeneous Pipelined Parallel-in-Time Framework
by: Huang, Ruixiang, et al.
Published: (2026)

Communication-Computation Pipeline Parallel Split Learning over Wireless Edge Networks
by: Liu, Chenyu, et al.
Published: (2025)

JanusPipe: Efficient Pipeline Parallel Training for Machine Learning Interatomic Potentials
by: Wang, Hongyu, et al.
Published: (2026)

TD-Pipe: Temporally-Disaggregated Pipeline Parallelism Architecture for High-Throughput LLM Inference
by: Zhang, Hongbin, et al.
Published: (2025)

Bandwidth-Aware and Cost-Efficient Pipeline Parallel Scheduling in Geo-Distributed LLM Training
by: Zhang, Han, et al.
Published: (2026)

Hecate: Unlocking Efficient Sparse Model Training via Fully Sharded Sparse Data Parallelism
by: Qing, Yuhao, et al.
Published: (2025)

SiPipe: Bridging the CPU-GPU Utilization Gap for Efficient Pipeline-Parallel LLM Inference
by: He, Yongchao, et al.
Published: (2025)

gLLM: Global Balanced Pipeline Parallelism System for Distributed LLM Serving with Token Throttling
by: Guo, Tianyu, et al.
Published: (2025)

SPPO:Efficient Long-sequence LLM Training via Adaptive Sequence Pipeline Parallel Offloading
by: Chen, Qiaoling, et al.
Published: (2025)

Enhancing Memory Efficiency in Large Language Model Training Through Chronos-aware Pipeline Parallelism
by: Lin, Xinyuan, et al.
Published: (2025)

Memory Efficient and Staleness Free Pipeline Parallel DNN Training Framework with Improved Convergence Speed
by: Dutta, Ankita, et al.
Published: (2025)

JUMBO: Fully Asynchronous BFT Consensus Made Truly Scalable
by: Cheng, Hao, et al.
Published: (2024)

MPipeMoE: Memory Efficient MoE for Pre-trained Models with Adaptive Pipeline Parallelism
by: Zhang, Zheng, et al.
Published: (2025)

TiMePReSt: Time and Memory Efficient Pipeline Parallel DNN Training with Removed Staleness
by: Dutta, Ankita, et al.
Published: (2024)

PPipe: Efficient Video Analytics Serving on Heterogeneous GPU Clusters via Pool-Based Pipeline Parallelism
by: Kong, Z. Jonny, et al.
Published: (2025)

SparseServe: Unlocking Parallelism for Dynamic Sparse Attention in Long-Context LLM Serving
by: Zhou, Qihui, et al.
Published: (2025)

Seq1F1B: Efficient Sequence-Level Pipeline Parallelism for Large Language Model Training
by: Sun, Ao, et al.
Published: (2024)