Copertina

Unveiling High-Probability Generalization in Decentralized SGD

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Wang, Jiahuan, Luo, Ping, Wen, Ziqing, Li, Dongsheng, Sun, Tao
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Stability and Generalization for Decentralized Markov SGD
di: Wang, Jiahuan, et al.
Pubblicazione: (2026)

FOAM: Blocked State Folding for Memory-Efficient LLM Training
di: Wen, Ziqing, et al.
Pubblicazione: (2025)

Federated Prediction-Powered Inference from Decentralized Data
di: Luo, Ping, et al.
Pubblicazione: (2024)

Local Gradient Regulation Stabilizes Federated Learning under Client Heterogeneity
di: Luo, Ping, et al.
Pubblicazione: (2026)

GWT: Scalable Optimizer State Compression for Large Language Model Training
di: Wen, Ziqing, et al.
Pubblicazione: (2025)

Score-based Generative Models with Adaptive Momentum
di: Wen, Ziqing, et al.
Pubblicazione: (2024)

Revealing Modular Gradient Noise Imbalance in LLMs: Calibrating Adam via Signal-to-Noise Ratio
di: Wen, Ziqing, et al.
Pubblicazione: (2026)

High-Probability Convergence Guarantees of Decentralized SGD
di: Armacki, Aleksandar, et al.
Pubblicazione: (2025)

Accelerating Federated Learning by Selecting Beneficial Herd of Local Gradients
di: Luo, Ping, et al.
Pubblicazione: (2024)

Topology-aware Generalization of Decentralized SGD
di: Zhu, Tongtian, et al.
Pubblicazione: (2022)

Improved Stability and Generalization Guarantees of the Decentralized SGD Algorithm
di: Bars, Batiste Le, et al.
Pubblicazione: (2023)

Heavy-Tail Phenomenon in Decentralized SGD
di: Gurbuzbalaban, Mert, et al.
Pubblicazione: (2022)

Differentially Private Clipped-SGD: High-Probability Convergence with Arbitrary Clipping Level
di: Khah, Saleh Vatan, et al.
Pubblicazione: (2025)

Minibatch and Local SGD: Algorithmic Stability and Linear Speedup in Generalization
di: Lei, Yunwen, et al.
Pubblicazione: (2023)

Tight Analysis of Decentralized SGD: A Markov Chain Perspective
di: Versini, Lucas, et al.
Pubblicazione: (2026)

Generalization and Optimization of SGD with Lookahead
di: Li, Kangcheng, et al.
Pubblicazione: (2025)

High-Probability Bounds for SGD under the Polyak-Lojasiewicz Condition with Markovian Noise
di: Kar, Avik, et al.
Pubblicazione: (2026)

Sharp High-Probability Rates for Nonlinear SGD under Heavy-Tailed Noise via Symmetrization
di: Armacki, Aleksandar, et al.
Pubblicazione: (2025)

Factor Augmented High-Dimensional SGD
di: Li, Shubo, et al.
Pubblicazione: (2026)

Towards Understanding Generalization and Stability Gaps between Centralized and Decentralized Federated Learning
di: Sun, Yan, et al.
Pubblicazione: (2023)

High-Probability Convergence in Decentralized Stochastic Optimization with Gradient Tracking
di: Armacki, Aleksandar, et al.
Pubblicazione: (2026)

Asynchronous Decentralized SGD under Non-Convexity: A Block-Coordinate Descent Framework
di: Zhou, Yijie, et al.
Pubblicazione: (2025)

Do Not Let Low-Probability Tokens Over-Dominate in RL for LLMs
di: Yang, Zhihe, et al.
Pubblicazione: (2025)

Adaptive debiased SGD in high-dimensional GLMs with streaming data
di: Han, Ruijian, et al.
Pubblicazione: (2024)

Leveraging Flatness to Improve Information-Theoretic Generalization Bounds for SGD
di: Peng, Ze, et al.
Pubblicazione: (2026)

Curvature-Informed SGD via General Purpose Lie-Group Preconditioners
di: Pooladzandi, Omead, et al.
Pubblicazione: (2024)

Where to Spend Rollouts: Hit-Utility Optimal Rollout Allocation for Group-Based RLVR
di: Wang, Tao, et al.
Pubblicazione: (2026)

Revisiting Gradient Normalization and Clipping for Nonconvex SGD under Heavy-Tailed Noise: Necessity, Sufficiency, and Acceleration
di: Sun, Tao, et al.
Pubblicazione: (2024)

Towards Understanding the Generalizability of Delayed Stochastic Gradient Descent
di: Deng, Xiaoge, et al.
Pubblicazione: (2023)

To Clip or not to Clip: the Dynamics of SGD with Gradient Clipping in High-Dimensions
di: Marshall, Noah, et al.
Pubblicazione: (2024)

Dual-Delayed Asynchronous SGD for Arbitrarily Heterogeneous Data
di: Wang, Xiaolu, et al.
Pubblicazione: (2024)

Convex SGD: Generalization Without Early Stopping
di: Hendrickx, Julien, et al.
Pubblicazione: (2024)

A Proportional-Integral Controller-Incorporated SGD Algorithm for High Efficient Latent Factor Analysis
di: Li, Jinli, et al.
Pubblicazione: (2025)

High-dimensional limit theorems for SGD: Momentum and Adaptive Step-sizes
di: Jagannath, Aukosh, et al.
Pubblicazione: (2025)

Unveiling the Power of Multiple Gossip Steps: A Stability-Based Generalization Analysis in Decentralized Training
di: Li, Qinglun, et al.
Pubblicazione: (2025)

The Optimality of (Accelerated) SGD for High-Dimensional Quadratic Optimization
di: Zhang, Haihan, et al.
Pubblicazione: (2024)

Data Deletion for Linear Regression with Noisy SGD
di: Xia, Zhangjie, et al.
Pubblicazione: (2024)

From Privacy to Generalization: Linear Max-Information Bounds for DP-SGD
di: Lampert, Christoph H., et al.
Pubblicazione: (2026)

MLFEF: Machine Learning Fusion Model with Empirical Formula to Explore the Momentum in Competitive Sports
di: Peng, Ruixin, et al.
Pubblicazione: (2024)

The Marginal Value of Momentum for Small Learning Rate SGD
di: Wang, Runzhe, et al.
Pubblicazione: (2023)