Image de couverture de livre

The Surprising Agreement Between Convex Optimization Theory and Learning-Rate Scheduling for Large Model Training

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Schaipp, Fabian, Hägele, Alexander, Taylor, Adrien, Simsekli, Umut, Bach, Francis
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Optimization and Control
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Optimization Benchmark for Diffusion Models on Dynamical Systems
par: Schaipp, Fabian
Publié: (2025)

Tracking the Median of Gradients with a Stochastic Proximal Point Method
par: Schaipp, Fabian, et autres
Publié: (2024)

Step-Size Stability in Stochastic Optimization: A Theoretical Perspective
par: Schaipp, Fabian, et autres
Publié: (2026)

A Convex Loss Function for Set Prediction with Optimal Trade-offs Between Size and Conditional Coverage
par: Bach, Francis
Publié: (2025)

Fast Stochastic Composite Minimization and an Accelerated Frank-Wolfe Algorithm under Parallelization
par: Dubois-Taine, Benjamin, et autres
Publié: (2022)

On the Effectiveness of the z-Transform Method in Quadratic Optimization
par: Bach, Francis
Publié: (2025)

Heavy-Tail Phenomenon in Decentralized SGD
par: Gurbuzbalaban, Mert, et autres
Publié: (2022)

A Tight Theory of Error Feedback Algorithms in Distributed Optimization
par: Thomsen, Daniel Berg, et autres
Publié: (2026)

Optimal Rates for Robust Stochastic Convex Optimization
par: Gao, Changyu, et autres
Publié: (2024)

Scaling Laws for Gradient Descent and Sign Descent for Linear Bigram Models under Zipf's Law
par: Kunstner, Frederik, et autres
Publié: (2025)

Efficient Optimization Algorithms for Linear Adversarial Training
par: RIbeiro, Antônio H., et autres
Publié: (2024)

Accelerated Rates between Stochastic and Adversarial Online Convex Optimization
par: Sachs, Sarah, et autres
Publié: (2023)

Online Convex Optimization and Integral Quadratic Constraints: An automated approach to regret analysis
par: Jakob, Fabian, et autres
Publié: (2025)

Algorithmic Stability of Stochastic Gradient Descent with Momentum under Heavy-Tailed Noise
par: Dang, Thanh, et autres
Publié: (2025)

Learning Rate Schedules in the Presence of Distribution Shift
par: Fahrbach, Matthew, et autres
Publié: (2023)

First-Order Sparse Convex Optimization: Better Rates with Sparse Updates
par: Garber, Dan
Publié: (2025)

Seesaw: Accelerating Training by Balancing Learning Rate and Batch Size Scheduling
par: Meterez, Alexandru, et autres
Publié: (2025)

A Spectral Framework for Closed-Form Relative Density Estimation
par: Bach, Francis
Publié: (2026)

Bilevel Optimization with Lower-Level Uniform Convexity: Theory and Algorithm
par: Wu, Yuman, et autres
Publié: (2026)

Are Convex Optimization Curves Convex?
par: Barzilai, Guy, et autres
Publié: (2025)

Some Primal-Dual Theory for Subgradient Methods for Strongly Convex Optimization
par: Grimmer, Benjamin, et autres
Publié: (2023)

A Semismooth Newton Stochastic Proximal Point Algorithm with Variance Reduction
par: Milzarek, Andre, et autres
Publié: (2022)

On the convergence rate of the boosted Difference-of-Convex Algorithm (DCA)
par: Abbaszadehpeivasti, Hadi, et autres
Publié: (2025)

On the Surprising Robustness of Sequential Convex Optimization for Contact-Implicit Motion Planning
par: Li, Yulin, et autres
Publié: (2025)

Deep Reinforcement Learning: A Convex Optimization Approach
par: Gattami, Ather
Publié: (2024)

Learning Algorithm Hyperparameters for Fast Parametric Convex Optimization
par: Sambharya, Rajiv, et autres
Publié: (2024)

Learning-Augmented Decentralized Online Convex Optimization in Networks
par: Li, Pengfei, et autres
Publié: (2023)

Optimization over Trained Neural Networks: Difference-of-Convex Algorithm and Application to Data Center Scheduling
par: Liu, Xinwei, et autres
Publié: (2025)

Anytime Training with Schedule-Free Spectral Optimization
par: Apte, Anuj, et autres
Publié: (2026)

Adversarial Training of Two-Layer Polynomial and ReLU Activation Networks via Convex Optimization
par: Kuelbs, Daniel, et autres
Publié: (2024)

Optimal Growth Schedules for Batch Size and Learning Rate in SGD that Reduce SFO Complexity
par: Umeda, Hikaru, et autres
Publié: (2025)

Variational Inference on the Boolean Hypercube with the Quantum Entropy
par: Beyler, Eliot, et autres
Publié: (2024)

Joint Learning in the Gaussian Single Index Model
par: Pillaud-Vivien, Loucas, et autres
Publié: (2025)

Improved Learning Rates for Stochastic Optimization
par: Li, Shaojie, et autres
Publié: (2021)

Stochastic Difference-of-Convex Optimization with Momentum
par: Chayti, El Mahdi, et autres
Publié: (2025)

The Price of Adaptivity in Stochastic Convex Optimization
par: Carmon, Yair, et autres
Publié: (2024)

Online Convex Optimization with Unbounded Memory
par: Kumar, Raunak, et autres
Publié: (2022)

Smooth Quasar-Convex Optimization with Constraints
par: Martínez-Rubio, David
Publié: (2025)

Unconstrained Robust Online Convex Optimization
par: Zhang, Jiujia, et autres
Publié: (2025)

Differentially Private Non-Convex Optimization under the KL Condition with Optimal Rates
par: Menart, Michael, et autres
Publié: (2023)