Cover Image

AdLoCo: adaptive batching significantly improves communications efficiency and convergence for Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kutuzov, Nikolay, Baderko, Makar, Kulibaba, Stepan, Dzhalilov, Artem, Bobrov, Daniel, Mashtaler, Maxim, Gasnikov, Alexander
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence Optimization and Control
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Exploring Applications of State Space Models and Advanced Training Techniques in Sequential Recommendations: A Comparative Study on Efficiency and Performance
by: Obozov, Mark, et al.
Published: (2024)

KompeteAI: Accelerated Autonomous Multi-Agent System for End-to-End Pipeline Generation for Machine Learning Problems
by: Kulibaba, Stepan, et al.
Published: (2025)

Lower and upper bounds of the convergence rate of gradient methods with composite noise in gradient
by: Vasin, Artem, et al.
Published: (2026)

SDG-MoE: Signed Debate Graph Mixture-of-Experts
by: Kulibaba, Stepan, et al.
Published: (2026)

Breaking the Heavy-Tailed Noise Barrier in Stochastic Optimization Problems
by: Puchkin, Nikita, et al.
Published: (2023)

Stochastic Decentralized Optimization of Non-Smooth Convex and Convex-Concave Problems over Time-Varying Networks
by: Divilkovskiy, Maxim, et al.
Published: (2025)

Median Clipping for Zeroth-order Non-Smooth Convex Optimization and Multi-Armed Bandit Problem with Heavy-tailed Symmetric Noise
by: Kornilov, Nikita, et al.
Published: (2024)

Advancing the lower bounds: An accelerated, stochastic, second-order method with optimal adaptation to inexactness
by: Agafonov, Artem, et al.
Published: (2023)

Exploiting higher-order derivatives in convex optimization methods
by: Kamzolov, Dmitry, et al.
Published: (2022)

OPTAMI: Global Superlinear Convergence of High-order Methods
by: Kamzolov, Dmitry, et al.
Published: (2024)

Power of Generalized Smoothness in Stochastic Convex Optimization: First- and Zero-Order Algorithms
by: Lobanov, Aleksandr, et al.
Published: (2025)

Accelerated Zero-Order SGD Method for Solving the Black Box Optimization Problem under "Overparametrization" Condition
by: Lobanov, Aleksandr, et al.
Published: (2023)

About some works of Boris Polyak on convergence of gradient methods and their development
by: Ablaev, Seydamet, et al.
Published: (2023)

Stochastic ADMM with batch size adaptation for nonconvex nonsmooth optimization
by: Jin, Jiachen, et al.
Published: (2025)

Decentralized Inexact Cubic Newton Method with Consensus Procedure
by: Agafonov, Artem, et al.
Published: (2026)

Simple Stepsize for Quasi-Newton Methods with Global Convergence Guarantees
by: Agafonov, Artem, et al.
Published: (2025)

Acceleration Exists! Optimization Problems When Oracle Can Only Compare Objective Function Values
by: Lobanov, Aleksandr, et al.
Published: (2024)

Strengthening the finite characterizations of smooth min-max games
by: Krivchenko, Valery, et al.
Published: (2026)

The Black-Box Optimization Problem: Zero-Order Accelerated Stochastic Method via Kernel Approximation
by: Lobanov, Aleksandr, et al.
Published: (2023)

Optimal Control of Nonlocal Balance Equations
by: Pogodaev, Nikolay, et al.
Published: (2025)

Optimal Analysis of Method with Batching for Monotone Stochastic Finite-Sum Variational Inequalities
by: Pichugin, Alexander, et al.
Published: (2024)

The Mirror-Prox Sliding Method for Non-smooth decentralized saddle-point problems
by: Kuruzov, Ilya, et al.
Published: (2022)

Exploring Jacobian Inexactness in Second-Order Methods for Variational Inequalities: Lower Bounds, Optimal Algorithms and Quasi-Newton Approximations
by: Agafonov, Artem, et al.
Published: (2024)

Decentralized Optimization with Mixed Affine Constraints
by: Yarmoshik, Demyan, et al.
Published: (2026)

Super-duality and necessary optimality conditions of order "infinity" in optimal control theory
by: Pogodaev, Nikolay, et al.
Published: (2025)

TS-RSR: A provably efficient approach for batch Bayesian Optimization
by: Ren, Zhaolin, et al.
Published: (2024)

Meta-learning for sample-efficient Bayesian optimisation of fed-batch processes
by: Langdon, Becky, et al.
Published: (2026)

A Parameter-free Decentralized Algorithm for Composite Convex Optimization
by: Chen, Xiaokai, et al.
Published: (2025)

Norm-Constrained Flows and Sign-Based Optimization: Theory and Algorithms
by: Leplat, Valentin, et al.
Published: (2025)

Linear Convergence Rate in Convex Setup is Possible! Gradient Descent Method Variants under $(L_0,L_1)$-Smoothness
by: Lobanov, Aleksandr, et al.
Published: (2024)

Extragradient Sliding for Composite Non-Monotone Variational Inequalities
by: Emelyanov, Roman, et al.
Published: (2024)

Nesterov's method of dichotomy via Order Oracle: The problem of optimizing a two-variable function on a square
by: Chervonenkis, Boris, et al.
Published: (2024)

Local SGD for Near-Quadratic Problems: Improving Convergence under Unconstrained Noise Conditions
by: Sadchikov, Andrey, et al.
Published: (2024)

Gradient-Type Methods For Decentralized Optimization Problems With Polyak-Łojasiewicz Condition Over Time-Varying Networks
by: Kuruzov, Ilya, et al.
Published: (2022)

Accelerated Methods with Compression for Horizontal and Vertical Federated Learning
by: Stanko, Sergey, et al.
Published: (2024)

Adaptive Stepsize Selection in Decentralized Convex Optimization
by: Kuruzov, Ilya, et al.
Published: (2025)

Universal methods for variational inequalities: deterministic and stochastic cases
by: Klimza, Anton, et al.
Published: (2024)

Method with Batching for Stochastic Finite-Sum Variational Inequalities in Non-Euclidean Setting
by: Pichugin, Alexander, et al.
Published: (2024)

Dual Smoothing for Decentralized Optimization
by: Rogozin, Alexander, et al.
Published: (2025)

Adaptive Algorithms for Relatively Lipschitz Continuous Convex Optimization Problems
by: Stonyakin, Fedor, et al.
Published: (2021)