Imagen de Portada

Nesterov Method for Asynchronous Pipeline Parallel Optimization

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Ajanthan, Thalaiyasingam, Ramasinghe, Sameera, Zuo, Yan, Avraham, Gil, Long, Alexander
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning Distributed, Parallel, and Cluster Computing
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

AsyncMesh: Fully Asynchronous Optimization for Data and Pipeline Parallelism
por: Ajanthan, Thalaiyasingam, et al.
Publicado: (2026)

SENTINEL: Stagewise Integrity Verification for Pipeline Parallel Decentralized Training
por: Dolatabadi, Hadi Mohaghegh, et al.
Publicado: (2026)

AMDP: Asynchronous Multi-Directional Pipeline Parallelism for Large-Scale Models Training
por: Chen, Ling, et al.
Publicado: (2026)

Protocol Models: Scaling Decentralized Training with Communication-Efficient Model Parallelism
por: Ramasinghe, Sameera, et al.
Publicado: (2025)

Mitigating Staleness in Asynchronous Pipeline Parallelism via Basis Rotation
por: Jung, Hyunji, et al.
Publicado: (2026)

Robust Fully-Asynchronous Methods for Distributed Training over General Architecture
por: Zhu, Zehan, et al.
Publicado: (2023)

PiPar: Pipeline Parallelism for Collaborative Machine Learning
por: Zhang, Zihan, et al.
Publicado: (2022)

HelixPipe: Efficient Distributed Training of Long Sequence Transformers with Attention Parallel Pipeline Parallelism
por: Zhang, Geng, et al.
Publicado: (2025)

AsyncHZP: Hierarchical ZeRO Parallelism with Asynchronous Scheduling for Scalable LLM Training
por: Bai, Huawei, et al.
Publicado: (2025)

Asynch-SGBDT: Asynchronous Parallel Stochastic Gradient Boosting Decision Tree based on Parameters Server
por: Daning, Cheng, et al.
Publicado: (2018)

A Readiness-Driven Runtime for Pipeline-Parallel Training under Runtime Variability
por: Liu, Ruitao, et al.
Publicado: (2026)

Pipeline Parallelism with Controllable Memory
por: Qi, Penghui, et al.
Publicado: (2024)

PipeLive: Efficient Live In-place Pipeline Parallelism Reconfiguration for Dynamic LLM Serving
por: Bai, Xu, et al.
Publicado: (2026)

A Tabular Schedule Abstraction for Communication-Aware Evaluation of Pipeline-Parallel LLM Training
por: Barley, Daniel, et al.
Publicado: (2026)

High-Dimensional Sparse Data Low-rank Representation via Accelerated Asynchronous Parallel Stochastic Gradient Descent
por: Hu, Qicong, et al.
Publicado: (2024)

PipeOffload: Improving Scalability of Pipeline Parallelism with Memory Optimization
por: Wan, Xinyi, et al.
Publicado: (2025)

On Optimizing the Communication of Model Parallelism
por: Zhuang, Yonghao, et al.
Publicado: (2022)

Asynchronous Federated Stochastic Optimization for Heterogeneous Objectives Under Arbitrary Delays
por: Iakovidou, Charikleia, et al.
Publicado: (2024)

PipeInfer: Accelerating LLM Inference using Asynchronous Pipelined Speculation
por: Butler, Branden, et al.
Publicado: (2024)

Asynchronous Personalized Federated Learning through Global Memorization
por: Wan, Fan, et al.
Publicado: (2025)

Straggler-Resilient Decentralized Learning via Adaptive Asynchronous Updates
por: Xiong, Guojun, et al.
Publicado: (2023)

FedQS: Optimizing Gradient and Model Aggregation for Semi-Asynchronous Federated Learning
por: Li, Yunbo, et al.
Publicado: (2025)

Zero Bubble Pipeline Parallelism
por: Qi, Penghui, et al.
Publicado: (2023)

Canzona: A Unified, Asynchronous, and Load-Balanced Framework for Distributed Matrix-based Optimizers
por: Wang, Liangyu, et al.
Publicado: (2026)

Re-evaluating the Memory-balanced Pipeline Parallelism: BPipe
por: Huang, Mincong, et al.
Publicado: (2024)

Ordered Momentum for Asynchronous SGD
por: Shi, Chang-Wei, et al.
Publicado: (2024)

Asynchronous Multi-Model Dynamic Federated Learning over Wireless Networks: Theory, Modeling, and Optimization
por: Chang, Zhan-Lun, et al.
Publicado: (2023)

Orthogonal Calibration for Asynchronous Federated Learning
por: Zhang, Jiayun, et al.
Publicado: (2025)

Birch SGD: A Tree Graph Framework for Local and Asynchronous SGD Methods
por: Tyurin, Alexander, et al.
Publicado: (2025)

AReaL-Hex: Accommodating Asynchronous RL Training over Heterogeneous GPUs
por: Yan, Ran, et al.
Publicado: (2025)

CaBaFL: Asynchronous Federated Learning via Hierarchical Cache and Feature Balance
por: Xia, Zeke, et al.
Publicado: (2024)

Asynchronous Federated Clustering with Unknown Number of Clusters
por: Zhang, Yunfan, et al.
Publicado: (2024)

FedAST: Federated Asynchronous Simultaneous Training
por: Askin, Baris, et al.
Publicado: (2024)

Learning to Shard: RL for Co-optimizing the Parallelism Degrees and Per-operator Sharding Dimensions in Distributed LLM Inference
por: Yin, Ruokai, et al.
Publicado: (2025)

Learning to Keep a Promise: Scaling Language Model Decoding Parallelism with Learned Asynchronous Decoding
por: Jin, Tian, et al.
Publicado: (2025)

Protocol Learning, Decentralized Frontier Risk and the No-Off Problem
por: Long, Alexander
Publicado: (2024)

Scaling Deep Learning Training with MPMD Pipeline Parallelism
por: Xhebraj, Anxhelo, et al.
Publicado: (2024)

DHO$_2$: Accelerating Distributed Hybrid Order Optimization via Model Parallelism and ADMM
por: Gu, Shunxian, et al.
Publicado: (2025)

Achieving Linear Speedup in Asynchronous Federated Learning with Heterogeneous Clients
por: Wang, Xiaolu, et al.
Publicado: (2024)

GAS: Generative Activation-Aided Asynchronous Split Federated Learning
por: Yang, Jiarong, et al.
Publicado: (2024)