Image de couverture de livre

Error Feedback under $(L_0,L_1)$-Smoothness: Normalization and Momentum

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Khirirat, Sarit, Sadiev, Abdurakhmon, Riabinin, Artem, Gorbunov, Eduard, Richtárik, Peter
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Better LMO-based Momentum Methods with Second-Order Information
par: Khirirat, Sarit, et autres
Publié: (2025)

Improved Convergence in Parameter-Agnostic Error Feedback through Momentum
par: Sadiev, Abdurakhmon, et autres
Publié: (2025)

Smoothed Normalization for Efficient Distributed Private Optimization
par: Shulgin, Egor, et autres
Publié: (2025)

A Unified Theory of Stochastic Proximal Point Methods without Smoothness
par: Richtárik, Peter, et autres
Publié: (2024)

Second-order Optimization under Heavy-Tailed Noise: Hessian Clipping and Sample Complexity Limits
par: Sadiev, Abdurakhmon, et autres
Publié: (2025)

Ringmaster LMO: Asynchronous Linear Minimization Oracle Momentum Method
par: Sadiev, Abdurakhmon, et autres
Publié: (2026)

SPAM: Stochastic Proximal Point Method with Momentum Variance Reduction for Non-convex Cross-Device Federated Learning
par: Karagulyan, Avetik, et autres
Publié: (2024)

Double Momentum and Error Feedback for Clipping with Fast Rates and Differential Privacy
par: Islamov, Rustem, et autres
Publié: (2025)

Differentially Private Random Block Coordinate Descent
par: Maranjyan, Artavazd, et autres
Publié: (2024)

Byzantine-Robust Optimization under $(L_0, L_1)$-Smoothness
par: Bolatov, Arman, et autres
Publié: (2026)

Methods for Convex $(L_0,L_1)$-Smooth Optimization: Clipping, Acceleration, and Adaptivity
par: Gorbunov, Eduard, et autres
Publié: (2024)

Collaborative Value Function Estimation Under Model Mismatch: A Federated Temporal Difference Analysis
par: Beikmohammadi, Ali, et autres
Publié: (2025)

High-Probability Convergence for Composite and Distributed Stochastic Minimization and Variational Inequalities with Heavy-Tailed Noise
par: Gorbunov, Eduard, et autres
Publié: (2023)

First Provable Guarantees for Practical Private FL: Beyond Restrictive Assumptions
par: Shulgin, Egor, et autres
Publié: (2025)

A Novel Unified Parametric Assumption for Nonconvex Optimization
par: Riabinin, Artem, et autres
Publié: (2025)

Parallel Momentum Methods Under Biased Gradient Estimations
par: Beikmohammadi, Ali, et autres
Publié: (2024)

EF21 with Bells & Whistles: Six Algorithmic Extensions of Modern Error Feedback
par: Fatkhullin, Ilyas, et autres
Publié: (2021)

Gluon: Making Muon & Scion Great Again! (Bridging Theory and Practice of LMO-based Optimizers for LLMs)
par: Riabinin, Artem, et autres
Publié: (2025)

Bernoulli-LoRA: A Theoretical Framework for Randomized Low-Rank Adaptation
par: Sokolov, Igor, et autres
Publié: (2025)

On the Interaction of Batch Noise, Adaptivity, and Compression, under $(L_0,L_1)$-Smoothness: An SDE Approach
par: Compagnoni, Enea Monzio, et autres
Publié: (2025)

Stochastic Proximal Point Methods for Monotone Inclusions under Expected Similarity
par: Sadiev, Abdurakhmon, et autres
Publié: (2024)

Convergence of Clipped-SGD for Convex $(L_0,L_1)$-Smooth Optimization with Heavy-Tailed Noise
par: Chezhegov, Savelii, et autres
Publié: (2025)

A Unified Primal-Dual Recipe for Accelerating Three-Operator Splitting Methods
par: Sadiev, Abdurakhmon, et autres
Publié: (2026)

A Nesterov-Accelerated Primal-Dual Splitting Algorithm for Convex Nonsmooth Optimization
par: Condat, Laurent, et autres
Publié: (2026)

Methods with Local Steps and Random Reshuffling for Generally Smooth Non-Convex Federated Optimization
par: Demidovich, Yury, et autres
Publié: (2024)

On the Convergence of Federated Learning Algorithms without Data Similarity
par: Beikmohammadi, Ali, et autres
Publié: (2024)

Compressed Federated Reinforcement Learning with a Generative Model
par: Beikmohammadi, Ali, et autres
Publié: (2024)

Byzantine Robustness and Partial Participation Can Be Achieved at Once: Just Clip Gradient Differences
par: Malinovsky, Grigory, et autres
Publié: (2023)

Byzantine-Robust and Differentially Private Federated Optimization under Weaker Assumptions
par: Islamov, Rustem, et autres
Publié: (2026)

Error Feedback for Muon and Friends
par: Gruntkowska, Kaja, et autres
Publié: (2025)

Error Feedback Reloaded: From Quadratic to Arithmetic Mean of Smoothness Constants
par: Richtárik, Peter, et autres
Publié: (2024)

Tight Lower Bounds and Optimal Algorithms for Stochastic Nonconvex Optimization with Heavy-Tailed Noise
par: Fradin, Adrien, et autres
Publié: (2025)

Communication Compression for Byzantine Robust Learning: New Efficient Algorithms and Improved Rates
par: Rammal, Ahmad, et autres
Publié: (2023)

Stochastic Gradient Methods with Preconditioned Updates
par: Sadiev, Abdurakhmon, et autres
Publié: (2022)

Muon is Provably Faster with Momentum Variance Reduction
par: Qian, Xun, et autres
Publié: (2025)

High Probability Complexity Bounds for Non-Smooth Stochastic Optimization with Heavy-Tailed Noise
par: Gorbunov, Eduard, et autres
Publié: (2021)

Where Does Warm-Up Come From? Adaptive Scheduling for Norm-Constrained Optimizers
par: Riabinin, Artem, et autres
Publié: (2026)

Linear Convergence Rate in Convex Setup is Possible! Gradient Descent Method Variants under $(L_0,L_1)$-Smoothness
par: Lobanov, Aleksandr, et autres
Publié: (2024)

Near-Optimal Convergence of Accelerated Gradient Methods under Generalized and $(L_0, L_1)$-Smoothness
par: Tyurin, Alexander
Publié: (2025)

Generalized Gradient Norm Clipping & Non-Euclidean $(L_0,L_1)$-Smoothness
par: Pethick, Thomas, et autres
Publié: (2025)