Copertina

HALoS: Hierarchical Asynchronous Local SGD over Slow Networks for Geo-Distributed Large Language Model Training

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Kim, Geon-Woo, Li, Junbo, Gandham, Shashidhar, Baldonado, Omar, Gangidi, Adithya, Balaji, Pavan, Wang, Zhangyang, Akella, Aditya
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Asynchronous Local-SGD Training for Language Modeling
di: Liu, Bo, et al.
Pubblicazione: (2024)

Read-ME: Refactorizing LLMs as Router-Decoupled Mixture of Experts with System Co-Design
di: Cai, Ruisi, et al.
Pubblicazione: (2024)

Pseudo-Asynchronous Local SGD: Robust and Efficient Data-Parallel Training
di: Naganuma, Hiroki, et al.
Pubblicazione: (2025)

A New Look At Small-$x$ Helicity Phenomenology: Investigating Uncertainties And The Impact Of Polarized Proton-proton Data
di: Baldonado, Nicholas
Pubblicazione: (2025)

Birch SGD: A Tree Graph Framework for Local and Asynchronous SGD Methods
di: Tyurin, Alexander, et al.
Pubblicazione: (2025)

On the Fundamental Limitations of Decentralized Learnable Reward Shaping in Cooperative Multi-Agent Reinforcement Learning
di: Akella, Aditya
Pubblicazione: (2025)

SLowcal-SGD: Slow Query Points Improve Local-SGD for Stochastic Convex Optimization
di: Dahan, Tehila, et al.
Pubblicazione: (2023)

Your Agents Are Aging Too: Agent Lifespan Engineering for Deployed Systems
di: Zhu, Jianing, et al.
Pubblicazione: (2026)

On-the-Fly Adaptive Distillation of Transformer to Dual-State Linear Attention
di: Ro, Yeonju, et al.
Pubblicazione: (2025)

HALoGEN: Fantastic LLM Hallucinations and Where to Find Them
di: Ravichander, Abhilasha, et al.
Pubblicazione: (2025)

Ordered Momentum for Asynchronous SGD
di: Shi, Chang-Wei, et al.
Pubblicazione: (2024)

Shadowheart SGD: Distributed Asynchronous SGD with Optimal Time Complexity Under Arbitrary Computation and Communication Heterogeneity
di: Tyurin, Alexander, et al.
Pubblicazione: (2024)

Improving the Throughput of Diffusion-based Large Language Models via a Training-Free Confidence-Aware Calibration
di: Shen, Jucheng, et al.
Pubblicazione: (2025)

A User-Centered Interface for Information Exploration in a Heterogeneous Digital Library.
di: Baldonado, Michelle Q. Wang
Pubblicazione: (2000)

Rescaled Asynchronous SGD: Optimal Distributed Optimization under Data and System Heterogeneity
di: Mahran, Ammar, et al.
Pubblicazione: (2026)

OMEGA: A Low-Latency GNN Serving System for Large Graphs
di: Kim, Geon-Woo, et al.
Pubblicazione: (2025)

BlockLLM: Multi-tenant Finer-grained Serving for Large Language Models
di: Hu, Bodun, et al.
Pubblicazione: (2024)

Comparative Analysis of Time Series Foundation Models for Demographic Forecasting: Enhancing Predictive Accuracy in US Population Dynamics
di: Akella, Aditya, et al.
Pubblicazione: (2025)

Robust Fully-Asynchronous Methods for Distributed Training over General Architecture
di: Zhu, Zehan, et al.
Pubblicazione: (2023)

Dual-Delayed Asynchronous SGD for Arbitrarily Heterogeneous Data
di: Wang, Xiaolu, et al.
Pubblicazione: (2024)

Delphi: Efficient Asynchronous Approximate Agreement for Distributed Oracles
di: Bandarupalli, Akhil, et al.
Pubblicazione: (2024)

When Do Graph Foundation Models Transfer? A Data-Centric Theory
di: Zhu, Jiajun, et al.
Pubblicazione: (2026)

Localizing Adversarial Attacks To Produces More Imperceptible Noise
di: Reddy, Pavan, et al.
Pubblicazione: (2025)

EDiT: A Local-SGD-Based Efficient Distributed Training Method for Large Language Models
di: Cheng, Jialiang, et al.
Pubblicazione: (2024)

PIPA: Preference Alignment as Prior-Informed Statistical Estimation
di: Li, Junbo, et al.
Pubblicazione: (2025)

Ringmaster ASGD: The First Asynchronous SGD with Optimal Time Complexity
di: Maranjyan, Artavazd, et al.
Pubblicazione: (2025)

First Provably Optimal Asynchronous SGD for Homogeneous and Heterogeneous Data
di: Maranjyan, Artavazd
Pubblicazione: (2026)

Demystifying Why Local Aggregation Helps: Convergence Analysis of Hierarchical SGD
di: Wang, Jiayi, et al.
Pubblicazione: (2020)

MindFlayer SGD: Efficient Parallel SGD in the Presence of Heterogeneous and Random Worker Compute Times
di: Maranjyan, Artavazd, et al.
Pubblicazione: (2024)

Spontaneous Baryogenesis and Primordial Black Hole Dark Matter from Ultra-Slow-Roll Inflation
di: Balaji, Shyam
Pubblicazione: (2026)

Do We Need Asynchronous SGD? On the Near-Optimality of Synchronous Methods
di: Begunov, Grigory, et al.
Pubblicazione: (2026)

The Iwasawa $μ$-invariants of Elliptic Curves over $\mathbb{Q}$
di: Chakravarthy, Adithya
Pubblicazione: (2024)

Exploring Scaling Laws for Local SGD in Large Language Model Training
di: He, Qiaozhi, et al.
Pubblicazione: (2024)

Asynchronous Local Computations in Distributed Bayesian Learning
di: Bhar, Kinjal, et al.
Pubblicazione: (2023)

The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication
di: Patel, Kumar Kshitij, et al.
Pubblicazione: (2024)

Asynchronous Training Schemes in Distributed Learning with Time Delay
di: Wang, Haoxiang, et al.
Pubblicazione: (2022)

Bringing Order to Asynchronous SGD: Towards Optimality under Data-Dependent Delays with Momentum
di: Dahan, Tehila, et al.
Pubblicazione: (2026)

Ringleader ASGD: The First Asynchronous SGD with Optimal Time Complexity under Data Heterogeneity
di: Maranjyan, Artavazd, et al.
Pubblicazione: (2025)

Asynchronous Decentralized SGD under Non-Convexity: A Block-Coordinate Descent Framework
di: Zhou, Yijie, et al.
Pubblicazione: (2025)

Sherlock: Reliable and Efficient Agentic Workflow Execution
di: Ro, Yeonju, et al.
Pubblicazione: (2025)