Imagen de Portada

Mitigating Staleness in Asynchronous Pipeline Parallelism via Basis Rotation

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Jung, Hyunji, Shin, Sungbin, Lee, Namhoon
Formato:	Preprint
Publicado:	2026
Materias:	Machine Learning Artificial Intelligence Distributed, Parallel, and Cluster Computing
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

A-3PO: Accelerating Asynchronous LLM Training with Staleness-aware Proximal Policy Approximation
por: Li, Xiaocan, et al.
Publicado: (2025)

FedStaleWeight: Buffered Asynchronous Federated Learning with Fair Aggregation via Staleness Reweighting
por: Ma, Jeffrey, et al.
Publicado: (2024)

Zero Bubble Pipeline Parallelism
por: Qi, Penghui, et al.
Publicado: (2023)

Nesterov Method for Asynchronous Pipeline Parallel Optimization
por: Ajanthan, Thalaiyasingam, et al.
Publicado: (2025)

Mitigating Persistent Client Dropout in Asynchronous Decentralized Federated Learning
por: Stępka, Ignacy, et al.
Publicado: (2025)

MSPipe: Efficient Temporal GNN Training via Staleness-Aware Pipeline
por: Sheng, Guangming, et al.
Publicado: (2024)

PipeOffload: Improving Scalability of Pipeline Parallelism with Memory Optimization
por: Wan, Xinyi, et al.
Publicado: (2025)

Piper: Efficient Large-Scale MoE Training via Resource Modeling and Pipelined Hybrid Parallelism
por: Dash, Sajal, et al.
Publicado: (2026)

ResBM: Residual Bottleneck Models for Low-Bandwidth Pipeline Parallelism
por: Aboudib, Alan, et al.
Publicado: (2026)

AsyncMesh: Fully Asynchronous Optimization for Data and Pipeline Parallelism
por: Ajanthan, Thalaiyasingam, et al.
Publicado: (2026)

GraphPipe: Improving Performance and Scalability of DNN Training with Graph Pipeline Parallelism
por: Jeon, Byungsoo, et al.
Publicado: (2024)

BitPipe: Bidirectional Interleaved Pipeline Parallelism for Accelerating Large Models Training
por: Wu, Houming, et al.
Publicado: (2024)

TawPipe: Topology-Aware Weight Pipeline Parallelism for Accelerating Long-Context Large Models Training
por: Wu, Houming, et al.
Publicado: (2025)

AMDP: Asynchronous Multi-Directional Pipeline Parallelism for Large-Scale Models Training
por: Chen, Ling, et al.
Publicado: (2026)

TimelyFreeze: Adaptive Parameter Freezing Mechanism for Pipeline Parallelism
por: Cho, Seonghye, et al.
Publicado: (2026)

Ravnest: Decentralized Asynchronous Training on Heterogeneous Devices
por: Menon, Anirudh Rajiv, et al.
Publicado: (2024)

Efficient Asynchronous Federated Learning with Sparsification and Quantization
por: Jia, Juncheng, et al.
Publicado: (2023)

GPRat: Gaussian Process Regression with Asynchronous Tasks
por: Helmann, Maksim, et al.
Publicado: (2025)

Laminar: A Scalable Asynchronous RL Post-Training Framework
por: Sheng, Guangming, et al.
Publicado: (2025)

Effective Heterogeneous Federated Learning via Efficient Hypernetwork-based Weight Generation
por: Shin, Yujin, et al.
Publicado: (2024)

FedFa: A Fully Asynchronous Training Paradigm for Federated Learning
por: Xu, Haotian, et al.
Publicado: (2024)

Online Parallel Multi-Task Relationship Learning via Alternating Direction Method of Multipliers
por: Li, Ruiyu, et al.
Publicado: (2024)

Semantic Parallelism: Redefining Efficient MoE Inference via Model-Data Co-Scheduling
por: Li, Yan, et al.
Publicado: (2025)

PIPO: Pipelined Offloading for Efficient Inference on Consumer Devices
por: Liu, Yangyijian, et al.
Publicado: (2025)

SEAFL: Enhancing Efficiency in Semi-Asynchronous Federated Learning through Adaptive Aggregation and Selective Training
por: Islam, Md Sirajul, et al.
Publicado: (2025)

Empirical Analysis of Asynchronous Federated Learning on Heterogeneous Devices: Efficiency, Fairness, and Privacy Trade-offs
por: Mohammadi, Samaneh, et al.
Publicado: (2025)

HybridEP: Scaling Expert Parallelism to Cross-Datacenter Scenario via Hybrid Expert/Data Transmission
por: Yang, Weihao, et al.
Publicado: (2025)

Parallel Split Learning with Global Sampling
por: Kohankhaki, Mohammad, et al.
Publicado: (2024)

DisagMoE: Computation-Communication overlapped MoE Training via Disaggregated AF-Pipe Parallelism
por: Zeng, Zhichen, et al.
Publicado: (2026)

Sampling Parallelism for Fast and Efficient Bayesian Learning
por: Özdemir, Asena Karolin, et al.
Publicado: (2026)

Context Parallelism for Scalable Million-Token Inference
por: Yang, Amy, et al.
Publicado: (2024)

Hermes: Memory-Efficient Pipeline Inference for Large Models on Edge Devices
por: Han, Xueyuan, et al.
Publicado: (2024)

Training Ultra Long Context Language Model with Fully Pipelined Distributed Transformer
por: Yao, Jinghan, et al.
Publicado: (2024)

MQ-GNN: A Multi-Queue Pipelined Architecture for Scalable and Efficient GNN Training
por: Ullah, Irfan, et al.
Publicado: (2026)

AdaPtis: Reducing Pipeline Bubbles with Adaptive Pipeline Parallelism on Heterogeneous Models
por: Guo, Jihu, et al.
Publicado: (2025)

Scalable and Adaptive Parallel Training of Graph Transformer on Large Graphs
por: Lin, Jun-Liang, et al.
Publicado: (2026)

FreeRide: Harvesting Bubbles in Pipeline Parallelism
por: Zhang, Jiashu, et al.
Publicado: (2024)

A Parallel Alternative for Energy-Efficient Neural Network Training and Inferencing
por: Seal, Sudip K., et al.
Publicado: (2025)

Tenplex: Dynamic Parallelism for Deep Learning using Parallelizable Tensor Collections
por: Wagenländer, Marcel, et al.
Publicado: (2023)

LlamaDuo: LLMOps Pipeline for Seamless Migration from Service LLMs to Small-Scale Local LLMs
por: Park, Chansung, et al.
Publicado: (2024)