Cover Image

An Adaptive Stochastic Gradient Method with Non-negative Gauss-Newton Stepsizes

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Orvieto, Antonio, Xiao, Lin
Format:	Preprint
Published:	2024
Subjects:	Optimization and Control Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Stochastic Approximation with Block Coordinate Optimal Stepsizes
by: Jiang, Tao, et al.
Published: (2025)

Simple Stepsize for Quasi-Newton Methods with Global Convergence Guarantees
by: Agafonov, Artem, et al.
Published: (2025)

Non-Asymptotic Optimization and Generalization Bounds for Stochastic Gauss-Newton in Overparameterized Models
by: Cayci, Semih
Published: (2025)

Coupling-based Convergence Diagnostic and Stepsize Scheme for Stochastic Gradient Descent
by: Li, Xiang, et al.
Published: (2024)

Dynamics of SGD with Stochastic Polyak Stepsizes: Truly Adaptive Variants and Convergence to Exact Solution
by: Orvieto, Antonio, et al.
Published: (2022)

Gauss-Newton Natural Gradient Descent for Shape Learning
by: King, James, et al.
Published: (2026)

Incremental Gauss--Newton Methods with Superlinear Convergence Rates
by: Zhou, Zhiling, et al.
Published: (2024)

Gradient Descent on Logistic Regression with Non-Separable Data and Large Step Sizes
by: Meng, Si Yi, et al.
Published: (2024)

MARINA-P: Superior Performance in Non-smooth Federated Optimization with Adaptive Stepsizes
by: Sokolov, Igor, et al.
Published: (2024)

Error whitening: Why Gauss-Newton outperforms Newton
by: McKay, Maricela Best, et al.
Published: (2026)

A Stochastic Quasi-Newton Method for Non-convex Optimization with Non-uniform Smoothness
by: Sun, Zhenyu, et al.
Published: (2024)

Stochastic Newton Proximal Extragradient Method
by: Jiang, Ruichen, et al.
Published: (2024)

Large Stepsize Gradient Descent for Non-Homogeneous Two-Layer Networks: Margin Improvement and Fast Optimization
by: Cai, Yuhang, et al.
Published: (2024)

Bias and Extrapolation in Markovian Linear Stochastic Approximation with Constant Stepsizes
by: Huo, Dongyan, et al.
Published: (2022)

Incremental Gauss-Newton Descent for Machine Learning
by: Korbit, Mikalai, et al.
Published: (2024)

Gradient Descent on Logistic Regression: Do Large Step-Sizes Work with Data on the Sphere?
by: Meng, Si Yi, et al.
Published: (2025)

AdaGrad Meets Muon: Adaptive Stepsizes for Orthogonal Updates
by: Zhang, Minxin, et al.
Published: (2025)

Enhancing Stochastic Optimization for Statistical Efficiency Using ROOT-SGD with Diminishing Stepsize
by: Li, Chris Junchi
Published: (2024)

Regularized Gauss-Newton for Optimizing Overparameterized Neural Networks
by: Adeoye, Adeyemi D., et al.
Published: (2024)

Stochastic Gradient Descent with Adaptive Data
by: Che, Ethan, et al.
Published: (2024)

Gauss-Newton Temporal Difference Learning with Nonlinear Function Approximation
by: Ke, Zhifa, et al.
Published: (2023)

On the Role of Batch Size in Stochastic Conditional Gradient Methods
by: Islamov, Rustem, et al.
Published: (2026)

The Collusion of Memory and Nonlinearity in Stochastic Approximation With Constant Stepsize
by: Huo, Dongyan, et al.
Published: (2024)

Two-Timescale Linear Stochastic Approximation: Constant Stepsizes Go a Long Way
by: Kwon, Jeongyeol, et al.
Published: (2024)

Is your batch size the problem? Revisiting the Adam-SGD gap in language modeling
by: Srećković, Teodora, et al.
Published: (2025)

Adaptive SGD with Line-Search and Polyak Stepsizes: Nonconvex Convergence and Accelerated Rates
by: Wu, Haotian
Published: (2025)

Quadratic Gradient: A Unified Framework Bridging Gradient Descent and Newton-Type Methods by Synthesizing Hessians and Gradients
by: Chiang, John
Published: (2022)

Unified Convergence Theory of Stochastic and Variance-Reduced Cubic Newton Methods
by: Chayti, El Mahdi, et al.
Published: (2023)

Stochastic Gradient Methods with Preconditioned Updates
by: Sadiev, Abdurakhmon, et al.
Published: (2022)

A Riemannian Optimization Perspective of the Gauss-Newton Method for Feedforward Neural Networks
by: Cayci, Semih
Published: (2024)

Stochastic Adaptive Gradient Descent Without Descent
by: Aujol, Jean-François, et al.
Published: (2025)

Recurrent neural networks: vanishing and exploding gradients are not the end of the story
by: Zucchet, Nicolas, et al.
Published: (2024)

Negative Stepsizes Make Gradient-Descent-Ascent Converge
by: Shugart, Henry, et al.
Published: (2025)

Complexity Lower Bounds of Adaptive Gradient Algorithms for Non-convex Stochastic Optimization under Relaxed Smoothness
by: Crawshaw, Michael, et al.
Published: (2025)

Adaptive Step Sizes for Preconditioned Stochastic Gradient Descent
by: Köhne, Frederik, et al.
Published: (2023)

Improving Stochastic Cubic Newton with Momentum
by: Chayti, El Mahdi, et al.
Published: (2024)

A Regularized Online Newton Method for Stochastic Convex Bandits with Linear Vanishing Noise
by: Zhan, Jingxin, et al.
Published: (2025)

Stochastic Non-Smooth Convex Optimization with Unbounded Gradients
by: Kovalev, Dmitry
Published: (2026)

Riemannian Stochastic Gradient Method for Nested Composition Optimization
by: Zhang, Dewei, et al.
Published: (2022)

Revisiting the Last-Iterate Convergence of Stochastic Gradient Methods
by: Liu, Zijian, et al.
Published: (2023)