Image de couverture de livre

Unveiling High-Probability Generalization in Decentralized SGD

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Wang, Jiahuan, Luo, Ping, Wen, Ziqing, Li, Dongsheng, Sun, Tao
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Stability and Generalization for Decentralized Markov SGD
par: Wang, Jiahuan, et autres
Publié: (2026)

FOAM: Blocked State Folding for Memory-Efficient LLM Training
par: Wen, Ziqing, et autres
Publié: (2025)

Federated Prediction-Powered Inference from Decentralized Data
par: Luo, Ping, et autres
Publié: (2024)

Local Gradient Regulation Stabilizes Federated Learning under Client Heterogeneity
par: Luo, Ping, et autres
Publié: (2026)

GWT: Scalable Optimizer State Compression for Large Language Model Training
par: Wen, Ziqing, et autres
Publié: (2025)

Score-based Generative Models with Adaptive Momentum
par: Wen, Ziqing, et autres
Publié: (2024)

Revealing Modular Gradient Noise Imbalance in LLMs: Calibrating Adam via Signal-to-Noise Ratio
par: Wen, Ziqing, et autres
Publié: (2026)

High-Probability Convergence Guarantees of Decentralized SGD
par: Armacki, Aleksandar, et autres
Publié: (2025)

Accelerating Federated Learning by Selecting Beneficial Herd of Local Gradients
par: Luo, Ping, et autres
Publié: (2024)

Topology-aware Generalization of Decentralized SGD
par: Zhu, Tongtian, et autres
Publié: (2022)

Improved Stability and Generalization Guarantees of the Decentralized SGD Algorithm
par: Bars, Batiste Le, et autres
Publié: (2023)

Heavy-Tail Phenomenon in Decentralized SGD
par: Gurbuzbalaban, Mert, et autres
Publié: (2022)

Differentially Private Clipped-SGD: High-Probability Convergence with Arbitrary Clipping Level
par: Khah, Saleh Vatan, et autres
Publié: (2025)

Minibatch and Local SGD: Algorithmic Stability and Linear Speedup in Generalization
par: Lei, Yunwen, et autres
Publié: (2023)

Tight Analysis of Decentralized SGD: A Markov Chain Perspective
par: Versini, Lucas, et autres
Publié: (2026)

Generalization and Optimization of SGD with Lookahead
par: Li, Kangcheng, et autres
Publié: (2025)

High-Probability Bounds for SGD under the Polyak-Lojasiewicz Condition with Markovian Noise
par: Kar, Avik, et autres
Publié: (2026)

Sharp High-Probability Rates for Nonlinear SGD under Heavy-Tailed Noise via Symmetrization
par: Armacki, Aleksandar, et autres
Publié: (2025)

Factor Augmented High-Dimensional SGD
par: Li, Shubo, et autres
Publié: (2026)

Towards Understanding Generalization and Stability Gaps between Centralized and Decentralized Federated Learning
par: Sun, Yan, et autres
Publié: (2023)

High-Probability Convergence in Decentralized Stochastic Optimization with Gradient Tracking
par: Armacki, Aleksandar, et autres
Publié: (2026)

Asynchronous Decentralized SGD under Non-Convexity: A Block-Coordinate Descent Framework
par: Zhou, Yijie, et autres
Publié: (2025)

Do Not Let Low-Probability Tokens Over-Dominate in RL for LLMs
par: Yang, Zhihe, et autres
Publié: (2025)

Adaptive debiased SGD in high-dimensional GLMs with streaming data
par: Han, Ruijian, et autres
Publié: (2024)

Leveraging Flatness to Improve Information-Theoretic Generalization Bounds for SGD
par: Peng, Ze, et autres
Publié: (2026)

Curvature-Informed SGD via General Purpose Lie-Group Preconditioners
par: Pooladzandi, Omead, et autres
Publié: (2024)

Where to Spend Rollouts: Hit-Utility Optimal Rollout Allocation for Group-Based RLVR
par: Wang, Tao, et autres
Publié: (2026)

Revisiting Gradient Normalization and Clipping for Nonconvex SGD under Heavy-Tailed Noise: Necessity, Sufficiency, and Acceleration
par: Sun, Tao, et autres
Publié: (2024)

Towards Understanding the Generalizability of Delayed Stochastic Gradient Descent
par: Deng, Xiaoge, et autres
Publié: (2023)

To Clip or not to Clip: the Dynamics of SGD with Gradient Clipping in High-Dimensions
par: Marshall, Noah, et autres
Publié: (2024)

Dual-Delayed Asynchronous SGD for Arbitrarily Heterogeneous Data
par: Wang, Xiaolu, et autres
Publié: (2024)

Convex SGD: Generalization Without Early Stopping
par: Hendrickx, Julien, et autres
Publié: (2024)

A Proportional-Integral Controller-Incorporated SGD Algorithm for High Efficient Latent Factor Analysis
par: Li, Jinli, et autres
Publié: (2025)

High-dimensional limit theorems for SGD: Momentum and Adaptive Step-sizes
par: Jagannath, Aukosh, et autres
Publié: (2025)

Unveiling the Power of Multiple Gossip Steps: A Stability-Based Generalization Analysis in Decentralized Training
par: Li, Qinglun, et autres
Publié: (2025)

The Optimality of (Accelerated) SGD for High-Dimensional Quadratic Optimization
par: Zhang, Haihan, et autres
Publié: (2024)

Data Deletion for Linear Regression with Noisy SGD
par: Xia, Zhangjie, et autres
Publié: (2024)

From Privacy to Generalization: Linear Max-Information Bounds for DP-SGD
par: Lampert, Christoph H., et autres
Publié: (2026)

MLFEF: Machine Learning Fusion Model with Empirical Formula to Explore the Momentum in Competitive Sports
par: Peng, Ruixin, et autres
Publié: (2024)

The Marginal Value of Momentum for Small Learning Rate SGD
par: Wang, Runzhe, et autres
Publié: (2023)