Cover Image

Non-Asymptotic Optimization and Generalization Bounds for Stochastic Gauss-Newton in Overparameterized Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Cayci, Semih
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Optimization and Control
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

A Riemannian Optimization Perspective of the Gauss-Newton Method for Feedforward Neural Networks
by: Cayci, Semih
Published: (2024)

Convergence of Stochastic Gradient Langevin Dynamics in the Lazy Training Regime
by: Oberweis, Noah, et al.
Published: (2025)

Regularized Gauss-Newton for Optimizing Overparameterized Neural Networks
by: Adeoye, Adeyemi D., et al.
Published: (2024)

Convergence of Gradient Descent for Recurrent Neural Networks: A Nonasymptotic Analysis
by: Cayci, Semih, et al.
Published: (2024)

Recurrent Natural Policy Gradient for POMDPs
by: Cayci, Semih, et al.
Published: (2024)

Linear Convergence of Entropy-Regularized Natural Policy Gradient with Linear Function Approximation
by: Cayci, Semih, et al.
Published: (2021)

An Adaptive Stochastic Gradient Method with Non-negative Gauss-Newton Stepsizes
by: Orvieto, Antonio, et al.
Published: (2024)

Finite-Time Analysis of Gradient Descent for Shallow Transformers
by: Arda, Enes, et al.
Published: (2026)

Error whitening: Why Gauss-Newton outperforms Newton
by: McKay, Maricela Best, et al.
Published: (2026)

A Stochastic Quasi-Newton Method for Non-convex Optimization with Non-uniform Smoothness
by: Sun, Zhenyu, et al.
Published: (2024)

Non-Asymptotic Bounds for Closed-Loop Identification of Unstable Nonlinear Stochastic Systems
by: Siriya, Seth, et al.
Published: (2024)

Incremental Gauss-Newton Descent for Machine Learning
by: Korbit, Mikalai, et al.
Published: (2024)

A Gauss-Newton Approach for Min-Max Optimization in Generative Adversarial Networks
by: Mishra, Neel, et al.
Published: (2024)

Optimal Rates of Convergence for Entropy Regularization in Discounted Markov Decision Processes
by: Müller, Johannes, et al.
Published: (2024)

Gauss-Newton Natural Gradient Descent for Shape Learning
by: King, James, et al.
Published: (2026)

Incremental Gauss--Newton Methods with Superlinear Convergence Rates
by: Zhou, Zhiling, et al.
Published: (2024)

Online Learning Guided Quasi-Newton Methods with Global Non-Asymptotic Convergence
by: Jiang, Ruichen, et al.
Published: (2024)

Gauss-Newton Temporal Difference Learning with Nonlinear Function Approximation
by: Ke, Zhifa, et al.
Published: (2023)

New Lower Bounds for Stochastic Non-Convex Optimization through Divergence Decomposition
by: Saad, El Mehdi, et al.
Published: (2025)

High Probability Complexity Bounds for Non-Smooth Stochastic Optimization with Heavy-Tailed Noise
by: Gorbunov, Eduard, et al.
Published: (2021)

Stochastic Newton Proximal Extragradient Method
by: Jiang, Ruichen, et al.
Published: (2024)

Improving Stochastic Cubic Newton with Momentum
by: Chayti, El Mahdi, et al.
Published: (2024)

Critical Influence of Overparameterization on Sharpness-aware Minimization
by: Shin, Sungbin, et al.
Published: (2023)

Complexity Lower Bounds of Adaptive Gradient Algorithms for Non-convex Stochastic Optimization under Relaxed Smoothness
by: Crawshaw, Michael, et al.
Published: (2025)

Provable Complexity Improvement of AdaGrad over SGD: Upper and Lower Bounds in Stochastic Non-Convex Optimization
by: Jiang, Ruichen, et al.
Published: (2024)

A Non-Asymptotic Theory of Seminorm Lyapunov Stability: From Deterministic to Stochastic Iterative Algorithms
by: Chen, Zaiwei, et al.
Published: (2025)

Implicit Regularization Makes Overparameterized Asymmetric Matrix Sensing Robust to Perturbations
by: Wind, Johan S.
Published: (2023)

Bayesian Optimization for Non-Convex Two-Stage Stochastic Optimization Problems
by: Buckingham, Jack M., et al.
Published: (2024)

Decentralized Non-convex Stochastic Optimization with Heterogeneous Variance
by: Chen, Hongxu, et al.
Published: (2026)

Online Non-Stationary Stochastic Quasar-Convex Optimization
by: Pun, Yuen-Man, et al.
Published: (2024)

Non-convex Stochastic Composite Optimization with Polyak Momentum
by: Gao, Yuan, et al.
Published: (2024)

Stochastic Non-Smooth Convex Optimization with Unbounded Gradients
by: Kovalev, Dmitry
Published: (2026)

Unified Convergence Theory of Stochastic and Variance-Reduced Cubic Newton Methods
by: Chayti, El Mahdi, et al.
Published: (2023)

Sharp Global Guarantees for Nonconvex Low-rank Recovery in the Noisy Overparameterized Regime
by: Zhang, Richard Y.
Published: (2021)

Improved Global Guarantees for the Nonconvex Burer--Monteiro Factorization via Rank Overparameterization
by: Zhang, Richard Y.
Published: (2022)

Fisher-Rao Gradient Flows of Linear Programs and State-Action Natural Policy Gradients
by: Müller, Johannes, et al.
Published: (2024)

Optimal Stochastic Non-smooth Non-convex Optimization through Online-to-Non-convex Conversion
by: Cutkosky, Ashok, et al.
Published: (2023)

Non-linear Multi-objective Optimization with Probabilistic Branch and Bound
by: Huang, Hao, et al.
Published: (2025)

RGNMR: A Gauss-Newton method for robust matrix completion with theoretical guarantees
by: Laufer, Eilon Vaknin, et al.
Published: (2025)

Towards Fully Parameter-Free Stochastic Optimization: Grid Search with Self-Bounding Analysis
by: Zhao, Yuheng, et al.
Published: (2026)