Cover Image

The High Line: Exact Risk and Learning Rate Curves of Stochastic Adaptive Learning Rate Algorithms

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Collins-Woodfin, Elizabeth, Seroussi, Inbar, Malaxechebarría, Begoña García, Mackenzie, Andrew W., Paquette, Elliot, Paquette, Courtney
Format:	Preprint
Published:	2024
Subjects:	Optimization and Control Statistics Theory Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Exact Dynamics of Multi-class Stochastic Gradient Descent
by: Collins-Woodfin, Elizabeth, et al.
Published: (2025)

High-dimensional Limit of SGD for Diagonal Linear Networks
by: Malaxechebarría, Begoña García, et al.
Published: (2026)

4+3 Phases of Compute-Optimal Neural Scaling Laws
by: Paquette, Elliot, et al.
Published: (2024)

Mirror Descent Algorithms with Nearly Dimension-Independent Rates for Differentially-Private Stochastic Saddle-Point Problems
by: González, Tomás, et al.
Published: (2024)

Dynamics of Stochastic Momentum with Sparse Updates in High Dimensions
by: Everett, Katie, et al.
Published: (2026)

Dimension-adapted Momentum Outscales SGD
by: Ferbach, Damien, et al.
Published: (2025)

Logarithmic-time Schedules for Scaling Language Models with Momentum
by: Ferbach, Damien, et al.
Published: (2026)

Phases of Muon: When Muon Eclipses SignSGD
by: Paquette, Elliot, et al.
Published: (2026)

Breaking a Logarithmic Barrier in the Stopping Time Convergence Rate of Stochastic First-order Methods
by: Feng, Yasong, et al.
Published: (2025)

Stochastic Optimal Control with Side Information and Bayesian Learning
by: Milz, Johannes, et al.
Published: (2026)

Non-asymptotic Global Convergence Rates of BFGS with Exact Line Search
by: Jin, Qiujiang, et al.
Published: (2024)

Sharp Rates of MMD Empirical Estimation with Power Kernels
by: Colasanto, Francesco, et al.
Published: (2026)

Risk-averse formulations of Stochastic Optimal Control and Markov Decision Processes
by: Shapiro, Alexander, et al.
Published: (2025)

Minimax Rates for Learning Pairwise Interactions in Attention-Style Models
by: Zucker, Shai, et al.
Published: (2025)

Adaptive Algorithms with Sharp Convergence Rates for Stochastic Hierarchical Optimization
by: Gong, Xiaochuan, et al.
Published: (2025)

Method of Moments for Estimation of Noisy Curves
by: Lo, Phillip, et al.
Published: (2024)

Improved Learning Rates for Stochastic Optimization
by: Li, Shaojie, et al.
Published: (2021)

On the Uniform Convergence of Subdifferentials in Stochastic Optimization and Learning
by: Ruan, Feng
Published: (2024)

Improved Rates for Stochastic Variance-Reduced Difference-of-Convex Algorithms
by: Nguyen, Anh Duc, et al.
Published: (2025)

Pessimism Meets Risk: Risk-Sensitive Offline Reinforcement Learning
by: Zhang, Dake, et al.
Published: (2024)

A Fundamental Convergence Rate Bound for Gradient Based Online Optimization Algorithms with Exact Tracking
by: Wu, Alex Xinting, et al.
Published: (2025)

Kernel Mean Embedding Topology: Weak and Strong Forms for Stochastic Kernels and Implications for Model Learning
by: Saldi, Naci, et al.
Published: (2025)

Convergence Rate of the Last Iterate of Stochastic Proximal Algorithms
by: Vaidyan, Kevin Kurian Thomas, et al.
Published: (2026)

The Essential Best and Average Rate of Convergence of the Exact Line Search Gradient Descent Method
by: Yu, Thomas
Published: (2023)

Stabilizing Rate of Stochastic Control Systems
by: Jia, Hui, et al.
Published: (2025)

Central Limit Theorems for Sample Average Approximations in Stochastic Optimal Control
by: Milz, Johannes, et al.
Published: (2025)

Wasserstein Proximal Coordinate Gradient Algorithms
by: Yao, Rentian, et al.
Published: (2024)

A Proof of the Exact Convergence Rate of Gradient Descent
by: Kim, Jungbin
Published: (2024)

Dyson Equation for Correlated Linearizations and Test Error of Random Features Regression
by: Latourelle-Vigeant, Hugo, et al.
Published: (2023)

Entropic Gromov-Wasserstein Distances: Stability and Algorithms
by: Rioux, Gabriel, et al.
Published: (2023)

Learning Rate Annealing Improves Tuning Robustness in Stochastic Optimization
by: Attia, Amit, et al.
Published: (2025)

Learning-Rate-Free Stochastic Optimization over Riemannian Manifolds
by: Dodd, Daniel, et al.
Published: (2024)

Adaptive Batch Size and Learning Rate Scheduler for Stochastic Gradient Descent Based on Minimization of Stochastic First-order Oracle Complexity
by: Umeda, Hikaru, et al.
Published: (2025)

Optimal Rates for Robust Stochastic Convex Optimization
by: Gao, Changyu, et al.
Published: (2024)

Adaptive discretization algorithms for locally optimal experimental design
by: Schmid, Jochen, et al.
Published: (2024)

Geometric Approach and Closed Exact Formulae for the Lasso
by: Dragović, Vladimir, et al.
Published: (2024)

On the Exactness of SDP Relaxation for Quadratic Assignment Problem
by: Ling, Shuyang
Published: (2024)

Risk Quadrangle and Robust Optimization Based on Extended $φ$-Divergence
by: Peng, Cheng, et al.
Published: (2024)

Statistical and Algorithmic Foundations of Reinforcement Learning
by: Chi, Yuejie, et al.
Published: (2025)

Risk-Aware Financial Forecasting Enhanced by Machine Learning and Intuitionistic Fuzzy Multi-Criteria Decision-Making
by: Turgay, Safiye, et al.
Published: (2025)