Copertina

Improved Convergence in Parameter-Agnostic Error Feedback through Momentum

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Sadiev, Abdurakhmon, Demidovich, Yury, Sokolov, Igor, Malinovsky, Grigory, Khirirat, Sarit, Richtárik, Peter
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Optimization and Control Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Better LMO-based Momentum Methods with Second-Order Information
di: Khirirat, Sarit, et al.
Pubblicazione: (2025)

A Unified Theory of Stochastic Proximal Point Methods without Smoothness
di: Richtárik, Peter, et al.
Pubblicazione: (2024)

Bernoulli-LoRA: A Theoretical Framework for Randomized Low-Rank Adaptation
di: Sokolov, Igor, et al.
Pubblicazione: (2025)

First Provable Guarantees for Practical Private FL: Beyond Restrictive Assumptions
di: Shulgin, Egor, et al.
Pubblicazione: (2025)

MAST: Model-Agnostic Sparsified Training
di: Demidovich, Yury, et al.
Pubblicazione: (2023)

Error Feedback under $(L_0,L_1)$-Smoothness: Normalization and Momentum
di: Khirirat, Sarit, et al.
Pubblicazione: (2024)

Second-order Optimization under Heavy-Tailed Noise: Hessian Clipping and Sample Complexity Limits
di: Sadiev, Abdurakhmon, et al.
Pubblicazione: (2025)

Ringmaster LMO: Asynchronous Linear Minimization Oracle Momentum Method
di: Sadiev, Abdurakhmon, et al.
Pubblicazione: (2026)

SPAM: Stochastic Proximal Point Method with Momentum Variance Reduction for Non-convex Cross-Device Federated Learning
di: Karagulyan, Avetik, et al.
Pubblicazione: (2024)

Differentially Private Random Block Coordinate Descent
di: Maranjyan, Artavazd, et al.
Pubblicazione: (2024)

Methods with Local Steps and Random Reshuffling for Generally Smooth Non-Convex Federated Optimization
di: Demidovich, Yury, et al.
Pubblicazione: (2024)

Smoothed Normalization for Efficient Distributed Private Optimization
di: Shulgin, Egor, et al.
Pubblicazione: (2025)

TAMUNA: Doubly Accelerated Distributed Optimization with Local Training, Compression, and Partial Participation
di: Condat, Laurent, et al.
Pubblicazione: (2023)

High-Probability Convergence for Composite and Distributed Stochastic Minimization and Variational Inequalities with Heavy-Tailed Noise
di: Gorbunov, Eduard, et al.
Pubblicazione: (2023)

EF21 with Bells & Whistles: Six Algorithmic Extensions of Modern Error Feedback
di: Fatkhullin, Ilyas, et al.
Pubblicazione: (2021)

A Unified Primal-Dual Recipe for Accelerating Three-Operator Splitting Methods
di: Sadiev, Abdurakhmon, et al.
Pubblicazione: (2026)

A Nesterov-Accelerated Primal-Dual Splitting Algorithm for Convex Nonsmooth Optimization
di: Condat, Laurent, et al.
Pubblicazione: (2026)

Stochastic Proximal Point Methods for Monotone Inclusions under Expected Similarity
di: Sadiev, Abdurakhmon, et al.
Pubblicazione: (2024)

MARINA-P: Superior Performance in Non-smooth Federated Optimization with Adaptive Stepsizes
di: Sokolov, Igor, et al.
Pubblicazione: (2024)

Double Momentum and Error Feedback for Clipping with Fast Rates and Differential Privacy
di: Islamov, Rustem, et al.
Pubblicazione: (2025)

Streamlining in the Riemannian Realm: Efficient Riemannian Optimization with Loopless Variance Reduction
di: Demidovich, Yury, et al.
Pubblicazione: (2024)

Tight Lower Bounds and Optimal Algorithms for Stochastic Nonconvex Optimization with Heavy-Tailed Noise
di: Fradin, Adrien, et al.
Pubblicazione: (2025)

Error Feedback for Muon and Friends
di: Gruntkowska, Kaja, et al.
Pubblicazione: (2025)

Byzantine Robustness and Partial Participation Can Be Achieved at Once: Just Clip Gradient Differences
di: Malinovsky, Grigory, et al.
Pubblicazione: (2023)

Correlated Quantization for Faster Nonconvex Distributed Optimization
di: Panferov, Andrei, et al.
Pubblicazione: (2024)

Byzantine-Robust and Differentially Private Federated Optimization under Weaker Assumptions
di: Islamov, Rustem, et al.
Pubblicazione: (2026)

Randomized Asymmetric Chain of LoRA: The First Meaningful Theoretical Framework for Low-Rank Adaptation
di: Malinovsky, Grigory, et al.
Pubblicazione: (2024)

Convergence Analysis of the PAGE Stochastic Algorithm for Weakly Convex Finite-Sum Optimization
di: Condat, Laurent, et al.
Pubblicazione: (2025)

Stochastic Gradient Methods with Preconditioned Updates
di: Sadiev, Abdurakhmon, et al.
Pubblicazione: (2022)

Muon is Provably Faster with Momentum Variance Reduction
di: Qian, Xun, et al.
Pubblicazione: (2025)

Revisiting Stochastic Proximal Point Methods: Generalized Smoothness and Similarity
di: Tovmasyan, Zhirayr, et al.
Pubblicazione: (2025)

Distance-Aware Muon: Adaptive Step Scaling for Normalized Optimization
di: Demidovich, Yury, et al.
Pubblicazione: (2026)

On the Convergence of DP-SGD with Adaptive Clipping
di: Shulgin, Egor, et al.
Pubblicazione: (2024)

Drop-Muon: Update Less, Converge Faster
di: Gruntkowska, Kaja, et al.
Pubblicazione: (2025)

Rennala MVR: Improved Time Complexity for Parallel Stochastic Optimization via Momentum-Based Variance Reduction
di: Tovmasyan, Zhirayr, et al.
Pubblicazione: (2026)

The Ball-Proximal (="Broximal") Point Method: a New Algorithm, Convergence Theory, and Applications
di: Gruntkowska, Kaja, et al.
Pubblicazione: (2025)

Improving the Worst-Case Bidirectional Communication Complexity for Nonconvex Distributed Optimization under Function Similarity
di: Gruntkowska, Kaja, et al.
Pubblicazione: (2024)

Non-Euclidean Broximal Point Method: A Blueprint for Geometry-Aware Optimization
di: Gruntkowska, Kaja, et al.
Pubblicazione: (2025)

A Computation and Communication Efficient Method for Distributed Nonconvex Problems in the Partial Participation Setting
di: Tyurin, Alexander, et al.
Pubblicazione: (2022)

Improving Convergence and Generalization Using Parameter Symmetries
di: Zhao, Bo, et al.
Pubblicazione: (2023)