Cover Image

Online Learning-guided Learning Rate Adaptation via Gradient Alignment

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Jiang, Ruichen, Kavis, Ali, Mokhtari, Aryan
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Optimization and Control
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Online Learning Guided Quasi-Newton Methods with Global Non-Asymptotic Convergence
by: Jiang, Ruichen, et al.
Published: (2024)

Adaptive and Optimal Second-order Optimistic Methods for Minimax Optimization
by: Jiang, Ruichen, et al.
Published: (2024)

Improved Complexity for Smooth Nonconvex Optimization: A Two-Level Online Learning Approach with Quasi-Newton Methods
by: Jiang, Ruichen, et al.
Published: (2024)

Improving Online-to-Nonconvex Conversion for Smooth Optimization via Double Optimism
by: Patitucci, Francisco, et al.
Published: (2025)

Adaptive Matrix Online Learning through Smoothing with Guarantees for Nonsmooth Nonconvex Optimization
by: Jiang, Ruichen, et al.
Published: (2026)

An Accelerated Gradient Method for Convex Smooth Simple Bilevel Optimization
by: Cao, Jincheng, et al.
Published: (2024)

Stochastic Newton Proximal Extragradient Method
by: Jiang, Ruichen, et al.
Published: (2024)

Provable Complexity Improvement of AdaGrad over SGD: Upper and Lower Bounds in Stochastic Non-Convex Optimization
by: Jiang, Ruichen, et al.
Published: (2024)

Adaptive Optimization via Momentum on Variance-Normalized Gradients
by: Patitucci, Francisco, et al.
Published: (2026)

Krylov Cubic Regularized Newton: A Subspace Second-Order Method with Dimension-Free Convergence Rate
by: Jiang, Ruichen, et al.
Published: (2024)

On the Complexity of Finding Stationary Points in Nonconvex Simple Bilevel Optimization
by: Cao, Jincheng, et al.
Published: (2025)

Generalized Optimistic Methods for Convex-Concave Saddle Point Problems
by: Jiang, Ruichen, et al.
Published: (2022)

Non-asymptotic Global Convergence Rates of BFGS with Exact Line Search
by: Jin, Qiujiang, et al.
Published: (2024)

An Inexact Conditional Gradient Method for Constrained Bilevel Optimization
by: Abolfazli, Nazanin, et al.
Published: (2023)

Provable and Practical Online Learning Rate Adaptation with Hypergradient Descent
by: Chu, Ya-Chi, et al.
Published: (2025)

Non-asymptotic Global Convergence Analysis of BFGS with the Armijo-Wolfe Line Search
by: Jin, Qiujiang, et al.
Published: (2024)

Gradient-Variation Online Learning under Generalized Smoothness
by: Xie, Yan-Feng, et al.
Published: (2024)

On the Crucial Role of Initialization for Matrix Factorization
by: Li, Bingcong, et al.
Published: (2024)

Universal Online Learning with Gradient Variations: A Multi-layer Online Ensemble Approach
by: Yan, Yu-Hu, et al.
Published: (2023)

Online Learning for Supervisory Switching Control
by: Sun, Haoyuan, et al.
Published: (2026)

Interpreting Adaptive Gradient Methods by Parameter Scaling for Learning-Rate-Free Optimization
by: Suh, Min-Kook, et al.
Published: (2024)

Increasing Both Batch Size and Learning Rate Accelerates Stochastic Gradient Descent
by: Umeda, Hikaru, et al.
Published: (2024)

AutoGD: Automatic Learning Rate Selection for Gradient Descent
by: Surjanovic, Nikola, et al.
Published: (2025)

Gradient Equilibrium in Online Learning: Theory and Applications
by: Angelopoulos, Anastasios N., et al.
Published: (2025)

Optimized Gradient Tracking for Decentralized Online Learning
by: Sharma, Shivangi Dubey, et al.
Published: (2023)

AutoSGD: Automatic Learning Rate Selection for Stochastic Gradient Descent
by: Surjanovic, Nikola, et al.
Published: (2025)

On the Convergence of Gradient Descent on Learning Transformers with Residual Connections
by: Qin, Zhen, et al.
Published: (2025)

Gradient Methods with Online Scaling
by: Gao, Wenzhi, et al.
Published: (2024)

Online (Non-)Convex Learning via Tempered Optimism
by: Haddouche, Maxime, et al.
Published: (2023)

An Energy-Based Self-Adaptive Learning Rate for Stochastic Gradient Descent: Enhancing Unconstrained Optimization with VAV method
by: Zhang, Jiahao, et al.
Published: (2024)

Rate-Optimal Regret for the Safe Learning-based Control of the Constrained Linear Quadratic Regulator
by: Hutchinson, Spencer, et al.
Published: (2026)

Federated Learning on Riemannian Manifolds: A Gradient-Free Projection-Based Approach
by: Wang, Hongye, et al.
Published: (2025)

Adaptive Batch Size and Learning Rate Scheduler for Stochastic Gradient Descent Based on Minimization of Stochastic First-order Oracle Complexity
by: Umeda, Hikaru, et al.
Published: (2025)

Active Learning of Deep Neural Networks via Gradient-Free Cutting Planes
by: Zhang, Erica, et al.
Published: (2024)

Reusing Historical Trajectories in Natural Policy Gradient via Importance Sampling: Convergence and Convergence Rate
by: Lin, Yifan, et al.
Published: (2024)

Bilevel Learning with Inexact Stochastic Gradients
by: Salehi, Mohammad Sadegh, et al.
Published: (2024)

Decision-Focused Learning with Directional Gradients
by: Huang, Michael, et al.
Published: (2024)

Open Problem: Anytime Convergence Rate of Gradient Descent
by: Kornowski, Guy, et al.
Published: (2024)

Fully Unconstrained Online Learning
by: Cutkosky, Ashok, et al.
Published: (2024)

Hessian-guided Perturbed Wasserstein Gradient Flows for Escaping Saddle Points
by: Yamamoto, Naoya, et al.
Published: (2025)