Cover Image

Bias-Variance Trade-off for Clipped Stochastic First-Order Methods: From Bounded Variance to Infinite Mean

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	He, Chuan
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence Optimization and Control Computation 49M37, 90C15, 90C25, 90C30, 90C90
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

A Proximal Stochastic Gradient Method with Adaptive Step Size and Variance Reduction for Convex Composite Optimization
by: Fang, Changjie, et al.
Published: (2025)

Level Constrained First Order Methods for Function Constrained Optimization
by: Boob, Digvijay, et al.
Published: (2022)

Variational Poisson Denoising via Augmented Lagrangian Methods
by: Kanzow, Christian, et al.
Published: (2023)

On the resolution of $\ell_1$-norm minimization via a two-metric adaptive projection method
by: Wu, Hanju, et al.
Published: (2025)

First- and Second-Order Stochastic Adaptive Regularization with Cubics: High Probability Iteration and Sample Complexity
by: Scheinberg, Katya, et al.
Published: (2023)

On Tackling High-Dimensional Nonconvex Stochastic Optimization via Stochastic First-Order Methods with Non-smooth Proximal Terms and Variance Reduction
by: Xie, Yue, et al.
Published: (2025)

Low-rank Orthogonalization for Large-scale Matrix Optimization with Applications to Foundation Model Training
by: He, Chuan, et al.
Published: (2025)

Retrospective Approximation Sequential Quadratic Programming for Stochastic Optimization with General Deterministic Nonlinear Constraints
by: Berahas, Albert S., et al.
Published: (2025)

On the Convergence and Complexity of Proximal Gradient and Accelerated Proximal Gradient Methods under Adaptive Gradient Estimation
by: Bollapragada, Raghu, et al.
Published: (2025)

Asymptotic Convergence and Stability of Adaptive Gradient Methods in Smooth Non-convex Optimization
by: Jin, Ruinan, et al.
Published: (2026)

A Structured Proximal Stochastic Variance Reduced Zeroth-order Algorithm
by: Rando, Marco, et al.
Published: (2025)

Augmented Lagrangian methods for fully convex composite optimization
by: De Marchi, Alberto, et al.
Published: (2025)

A Newton Augmented Lagrangian Method for Symmetric Cone Programming with Complexity Analysis
by: Zhang, Rui-Jin, et al.
Published: (2025)

The Grimmer--Shu--Wang Certificate and the Drori--Teboulle Minimax Constant-Stepsize Bound for $N\ge 3$
by: Zhang, Lixing
Published: (2026)

Dual dynamic programming for stochastic programs over an infinite horizon
by: Ju, Caleb, et al.
Published: (2023)

A stochastic first-order method with multi-extrapolated momentum for highly smooth unconstrained optimization
by: He, Chuan
Published: (2024)

On the Convergence of Interior-Point Methods for Bound-Constrained Nonlinear Optimization Problems with Noise
by: Dezfulian, Shima, et al.
Published: (2024)

Exploiting cone approximations in an augmented Lagrangian method for conic optimization
by: Fukuda, Mituhiro, et al.
Published: (2024)

Modified limited memory BFGS with displacement aggregation and its application to the largest eigenvalue problem
by: Sahu, Manish Kumar, et al.
Published: (2023)

A Two Stepsize SQP Method for Nonlinear Equality Constrained Stochastic Optimization
by: O'Neill, Michael J.
Published: (2024)

Accelerated stochastic first-order method for convex optimization under heavy-tailed noise
by: He, Chuan, et al.
Published: (2025)

A cut-and-project perspective for linearized Bregman iterations
by: Dai, Yu-Hong, et al.
Published: (2024)

Bregman Stochastic Proximal Point Algorithm with Variance Reduction
by: Traoré, Cheik, et al.
Published: (2025)

MGProx: A nonsmooth multigrid proximal gradient method with adaptive restriction for strongly convex optimization
by: Ang, Andersen, et al.
Published: (2023)

Stochastic First-Order Methods with Non-smooth and Non-Euclidean Proximal Terms for Nonconvex High-Dimensional Stochastic Optimization
by: Xie, Yue, et al.
Published: (2024)

Optimal Subgradient Methods for Lipschitz Convex Optimization with Error Bounds
by: Wang, Alex L.
Published: (2025)

Cubic Regularization Technique of the Newton Method for Vector Optimization
by: Ghosh, Debdas
Published: (2025)

Trust-Region Method for Optimization of Set-Valued Maps Given by Finitely Many Functions
by: Ghosh, Suprova, et al.
Published: (2025)

Stochastic Variance-Reduced Forward-Reflected-Backward Splitting Methods for Nonmonotone Generalized Equations
by: Tran-Dinh, Quoc
Published: (2024)

A Globally Convergent Method for Computing B-stationary Points of Mathematical Programs with Equilibrium Constraints
by: Nurkanović, Armin, et al.
Published: (2025)

A Stochastic Quasi-Newton Method in the Absence of Common Random Numbers
by: Menickelly, Matt, et al.
Published: (2023)

A Newton-CG based barrier-augmented Lagrangian method for general nonconvex conic optimization
by: He, Chuan, et al.
Published: (2023)

Operation of an ambulance fleet under uncertainty
by: Guigues, Vincent, et al.
Published: (2022)

Parallel block coordinate descent methods with identification strategies
by: Lopes, Ronaldo, et al.
Published: (2025)

A Note on Piecewise Affine Decision Rules for Robust, Stochastic, and Data-Driven Optimization
by: Thomä, Simon, et al.
Published: (2024)

An Inexact Regularized Proximal Newton Method without Line Search
by: Dahl, Simeon vom, et al.
Published: (2024)

Local Convergence Results for Sequential Quadratic Programming with Complementarity Constraints
by: Nurkanović, Armin
Published: (2026)

Parallelizing the Circumcentered-Reflection Method
by: Barros, Pablo, et al.
Published: (2025)

On Convergence of Regularized Barzilai-Borwein Method
by: Xu, Xin
Published: (2025)

A Modified Proximal Bundle Method Under A Frank-Wolfe Perspective
by: Fersztand, David, et al.
Published: (2024)