Image de couverture de livre

Avoiding Bias in Clipped SGD for Overparameterized Models under Generalized Smoothness

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Lobanov, Aleksandr, Koloskova, Anastasia
Format:	Preprint
Publié:	2026
Sujets:	Optimization and Control
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Improved Iteration Complexity in Black-Box Optimization Problems under Higher Order Smoothness Function Condition
par: Lobanov, Aleksandr
Publié: (2024)

Accelerated Zero-Order SGD Method for Solving the Black Box Optimization Problem under "Overparametrization" Condition
par: Lobanov, Aleksandr, et autres
Publié: (2023)

Power of Generalized Smoothness in Stochastic Convex Optimization: First- and Zero-Order Algorithms
par: Lobanov, Aleksandr, et autres
Publié: (2025)

Convergence of Clipped-SGD for Convex $(L_0,L_1)$-Smooth Optimization with Heavy-Tailed Noise
par: Chezhegov, Savelii, et autres
Publié: (2025)

Accelerated zero-order SGD under high-order smoothness and overparameterized regime
par: Bychkov, Georgii, et autres
Publié: (2024)

Linear Convergence Rate in Convex Setup is Possible! Gradient Descent Method Variants under $(L_0,L_1)$-Smoothness
par: Lobanov, Aleksandr, et autres
Publié: (2024)

Median Clipping for Zeroth-order Non-Smooth Convex Optimization and Multi-Armed Bandit Problem with Heavy-tailed Symmetric Noise
par: Kornilov, Nikita, et autres
Publié: (2024)

Convergence of Clipped SGD on Convex $(L_0,L_1)$-Smooth Functions
par: Gaash, Ofir, et autres
Publié: (2025)

On Convergence of Incremental Gradient for Non-Convex Smooth Functions
par: Koloskova, Anastasia, et autres
Publié: (2023)

Clipped SGD Algorithms for Performative Prediction: Tight Bounds for Clipping Bias and Remedies
par: Li, Qiang, et autres
Publié: (2024)

FedMuon: Federated Learning with Bias-corrected LMO-based Optimization
par: Takezawa, Yuki, et autres
Publié: (2025)

Differentially Private Clipped-SGD: High-Probability Convergence with Arbitrary Clipping Level
par: Khah, Saleh Vatan, et autres
Publié: (2025)

Acceleration Exists! Optimization Problems When Oracle Can Only Compare Objective Function Values
par: Lobanov, Aleksandr, et autres
Publié: (2024)

The Black-Box Optimization Problem: Zero-Order Accelerated Stochastic Method via Kernel Approximation
par: Lobanov, Aleksandr, et autres
Publié: (2023)

Gradient-free algorithm for saddle point problems under overparametrization
par: Statkevich, Ekaterina, et autres
Publié: (2024)

On the Convergence of DP-SGD with Adaptive Clipping
par: Shulgin, Egor, et autres
Publié: (2024)

From Gradient Clipping to Normalization for Heavy Tailed SGD
par: Hübler, Florian, et autres
Publié: (2024)

Local SGD for Near-Quadratic Problems: Improving Convergence under Unconstrained Noise Conditions
par: Sadchikov, Andrey, et autres
Publié: (2024)

Clipped Stochastic Gradient Tracking For Locally Smooth Functions
par: Mei, Leilei, et autres
Publié: (2026)

Revisiting Gradient Normalization and Clipping for Nonconvex SGD under Heavy-Tailed Noise: Necessity, Sufficiency, and Acceleration
par: Sun, Tao, et autres
Publié: (2024)

Nesterov's method of dichotomy via Order Oracle: The problem of optimizing a two-variable function on a square
par: Chervonenkis, Boris, et autres
Publié: (2024)

Last-Iterate Complexity of SGD for Convex and Smooth Stochastic Problems
par: Garrigos, Guillaume, et autres
Publié: (2025)

Tight Long-Term Tail Decay of (Clipped) SGD in Non-Convex Optimization
par: Armacki, Aleksandar, et autres
Publié: (2026)

About some works of Boris Polyak on convergence of gradient methods and their development
par: Ablaev, Seydamet, et autres
Publié: (2023)

Optimization Outperforms Unscented Techniques for Nonlinear Smoothing
par: Howell, Payton, et autres
Publié: (2025)

Diagonalisation SGD: Fast & Convergent SGD for Non-Differentiable Models via Reparameterisation and Smoothing
par: Wagner, Dominik, et autres
Publié: (2024)

The Rich and the Simple: On the Implicit Bias of Adam and SGD
par: Vasudeva, Bhavya, et autres
Publié: (2025)

Randomized gradient-free methods in convex optimization
par: Gasnikov, Alexander, et autres
Publié: (2022)

Fast Last-Iterate Convergence of SGD in the Smooth Interpolation Regime
par: Attia, Amit, et autres
Publié: (2025)

Non-Asymptotic Optimization and Generalization Bounds for Stochastic Gauss-Newton in Overparameterized Models
par: Cayci, Semih
Publié: (2025)

Sign-SGD via Parameter-Free Optimization
par: Medyakov, Daniil, et autres
Publié: (2025)

Smoothed Gradient Clipping and Error Feedback for Decentralized Optimization under Symmetric Heavy-Tailed Noise
par: Yu, Shuhua, et autres
Publié: (2023)

Nonlinearly Preconditioned Gradient Methods under Generalized Smoothness
par: Oikonomidis, Konstantinos, et autres
Publié: (2025)

Methods for Convex $(L_0,L_1)$-Smooth Optimization: Clipping, Acceleration, and Adaptivity
par: Gorbunov, Eduard, et autres
Publié: (2024)

A Proximal Method for Composite Optimization with Smooth and Convex Components
par: Uzun, Samet, et autres
Publié: (2025)

On the (almost) Global Exponential Convergence of the Overparameterized Policy Optimization for the LQR Problem
par: Wafi, Moh Kamalul, et autres
Publié: (2025)

Bias-Optimal Bounds for SGD: A Computer-Aided Lyapunov Analysis
par: Cortild, Daniel, et autres
Publié: (2025)

An Improved Analysis of the Clipped Stochastic subGradient Method under Heavy-Tailed Noise
par: Parletta, Daniela Angela, et autres
Publié: (2024)

Near-Optimal Tensor PCA via Normalized Stochastic Gradient Ascent with Overparameterization
par: Ding, Shihong, et autres
Publié: (2025)

Toward a Unified Theory of Gradient Descent under Generalized Smoothness
par: Tyurin, Alexander
Publié: (2024)