Imagen de Portada

Armijo Line-search Can Make (Stochastic) Gradient Descent Provably Faster

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Vaswani, Sharan, Babanezhad, Reza
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning Optimization and Control
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Towards Noise-adaptive, Problem-adaptive (Accelerated) Stochastic Gradient Descent
por: Vaswani, Sharan, et al.
Publicado: (2021)

Convergence of Steepest Descent and Adam under Non-Uniform Smoothness
por: Vaswani, Sharan, et al.
Publicado: (2026)

(Accelerated) Noise-adaptive Stochastic Heavy-Ball Momentum
por: Dang, Anh, et al.
Publicado: (2024)

Glocal Smoothness: Line search and adaptive step sizes can help in theory too!
por: Fox, Curtis, et al.
Publicado: (2025)

Relationship between Batch Size and Number of Steps Needed for Nonconvex Optimization of Stochastic Gradient Descent using Armijo Line Search
por: Tsukada, Yuki, et al.
Publicado: (2023)

Provably Faster Gradient Descent via Long Steps
por: Grimmer, Benjamin
Publicado: (2023)

Faster Convergence of Stochastic Accelerated Gradient Descent under Interpolation
por: Mishkin, Aaron, et al.
Publicado: (2024)

Faster Convergence of Riemannian Stochastic Gradient Descent with Increasing Batch Size
por: Oowada, Kanata, et al.
Publicado: (2025)

On the Provable Suboptimality of Momentum SGD in Nonstationary Stochastic Optimization
por: Sahu, Sharan, et al.
Publicado: (2026)

On the Inherent Privacy of Zeroth Order Projected Gradient Descent
por: Gupta, Devansh, et al.
Publicado: (2025)

From Inverse Optimization to Feasibility to ERM
por: Mishra, Saurabh, et al.
Publicado: (2024)

Learning Provably Improves the Convergence of Gradient Descent
por: Song, Qingyu, et al.
Publicado: (2025)

Stochastic Adaptive Gradient Descent Without Descent
por: Aujol, Jean-François, et al.
Publicado: (2025)

Stochastic Gradient Descent with Strategic Querying
por: Jiang, Nanfei, et al.
Publicado: (2025)

Stochastic Gradient Descent with Adaptive Data
por: Che, Ethan, et al.
Publicado: (2024)

Momentum Does Not Reduce Stochastic Noise in Stochastic Gradient Descent
por: Sato, Naoki, et al.
Publicado: (2024)

Derivatives of Stochastic Gradient Descent in parametric optimization
por: Iutzeler, Franck, et al.
Publicado: (2024)

Enhancing Stochastic Gradient Descent: A Unified Framework and Novel Acceleration Methods for Faster Convergence
por: Deng, Yichuan, et al.
Publicado: (2024)

Faster Acceleration for Steepest Descent
por: Bai, Cedar Site, et al.
Publicado: (2024)

Convergence Analysis of Stochastic Gradient Descent with MCMC Estimators
por: Li, Tianyou, et al.
Publicado: (2023)

The Sample Complexity of Gradient Descent in Stochastic Convex Optimization
por: Livni, Roi
Publicado: (2024)

Adaptive Step Sizes for Preconditioned Stochastic Gradient Descent
por: Köhne, Frederik, et al.
Publicado: (2023)

Functional Central Limit Theorem for Stochastic Gradient Descent
por: Flamand, Kessang, et al.
Publicado: (2026)

Parameter Symmetry and Noise Equilibrium of Stochastic Gradient Descent
por: Ziyin, Liu, et al.
Publicado: (2024)

Faster Gradient Methods for Highly-Smooth Stochastic Bilevel Optimization
por: Chen, Lesi, et al.
Publicado: (2025)

Faster Gradient-Free Algorithms for Nonsmooth Nonconvex Stochastic Optimization
por: Chen, Lesi, et al.
Publicado: (2023)

Rapid Overfitting of Multi-Pass Stochastic Gradient Descent in Stochastic Convex Optimization
por: Vansover-Hager, Shira, et al.
Publicado: (2025)

Muon is Provably Faster with Momentum Variance Reduction
por: Qian, Xun, et al.
Publicado: (2025)

Implicit Bias and Convergence of Matrix Stochastic Mirror Descent
por: Akhtiamov, Danil, et al.
Publicado: (2026)

Exponential Convergence of (Stochastic) Gradient Descent for Separable Logistic Regression
por: Kale, Sacchit, et al.
Publicado: (2026)

On the Convergence of Stochastic Gradient Descent with Perturbed Forward-Backward Passes
por: Kong, Boao, et al.
Publicado: (2026)

Stochastic Subspace Descent Accelerated via Bi-fidelity Line Search
por: Cheng, Nuojin, et al.
Publicado: (2025)

Dissecting Discrete Soft Actor-Critic: Limitations and Principled Alternatives
por: Asad, Reza, et al.
Publicado: (2025)

Coupling-based Convergence Diagnostic and Stepsize Scheme for Stochastic Gradient Descent
por: Li, Xiang, et al.
Publicado: (2024)

Decentralized Stochastic Gradient Descent Ascent for Finite-Sum Minimax Problems
por: Gao, Hongchang
Publicado: (2022)

Mitigating Gradient Bias in Multi-objective Learning: A Provably Convergent Stochastic Approach
por: Fernando, Heshan, et al.
Publicado: (2022)

First and Second Order Approximations to Stochastic Gradient Descent Methods with Momentum Terms
por: Lu, Eric
Publicado: (2025)

GeoAdaLer: Geometric Insights into Adaptive Stochastic Gradient Descent Algorithms
por: Eleh, Chinedu, et al.
Publicado: (2024)

Increasing Both Batch Size and Learning Rate Accelerates Stochastic Gradient Descent
por: Umeda, Hikaru, et al.
Publicado: (2024)

FastPart: Over-Parameterized Stochastic Gradient Descent for Sparse optimisation on Measures
por: De Castro, Yohann, et al.
Publicado: (2023)