Copertina

Unbiased and Sign Compression in Distributed Learning: Comparing Noise Resilience via SDEs

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Compagnoni, Enea Monzio, Islamov, Rustem, Proske, Frank Norbert, Lucchi, Aurelien
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Adaptive Methods through the Lens of SDEs: Theoretical Insights on the Role of Noise
di: Compagnoni, Enea Monzio, et al.
Pubblicazione: (2024)

On the Interaction of Batch Noise, Adaptivity, and Compression, under $(L_0,L_1)$-Smoothness: An SDE Approach
di: Compagnoni, Enea Monzio, et al.
Pubblicazione: (2025)

SDEs for Minimax Optimization
di: Compagnoni, Enea Monzio, et al.
Pubblicazione: (2024)

Adaptive Methods Are Preferable in High Privacy Settings: An SDE Perspective
di: Compagnoni, Enea Monzio, et al.
Pubblicazione: (2026)

Why Do We Need Warm-up? A Theoretical Perspective
di: Alimisis, Foivos, et al.
Pubblicazione: (2025)

Enhancing Optimizer Stability: Momentum Adaptation of The NGN Step-size
di: Islamov, Rustem, et al.
Pubblicazione: (2025)

Loss Landscape Characterization of Neural Networks without Over-Parametrization
di: Islamov, Rustem, et al.
Pubblicazione: (2024)

Double Momentum and Error Feedback for Clipping with Fast Rates and Differential Privacy
di: Islamov, Rustem, et al.
Pubblicazione: (2025)

Byzantine-Robust and Differentially Private Federated Optimization under Weaker Assumptions
di: Islamov, Rustem, et al.
Pubblicazione: (2026)

On the Role of Batch Size in Stochastic Conditional Gradient Methods
di: Islamov, Rustem, et al.
Pubblicazione: (2026)

Towards Faster Decentralized Stochastic Optimization with Communication Compression
di: Islamov, Rustem, et al.
Pubblicazione: (2024)

Safe-EF: Error Feedback for Nonsmooth Constrained Optimization
di: Islamov, Rustem, et al.
Pubblicazione: (2025)

Non-Euclidean Gradient Descent Operates at the Edge of Stability
di: Islamov, Rustem, et al.
Pubblicazione: (2026)

A Theoretical Analysis of the Learning Dynamics under Class Imbalance
di: Francazi, Emanuele, et al.
Pubblicazione: (2022)

Unpacking Softmax: How Temperature Drives Representation Collapse, Compression, and Generalization
di: Masarczyk, Wojciech, et al.
Pubblicazione: (2025)

On the Intrinsic Dimensions of Data in Kernel Learning
di: Takhanov, Rustem
Pubblicazione: (2026)

A Comprehensive Analysis on the Learning Curve in Kernel Ridge Regression
di: Cheng, Tin Sum, et al.
Pubblicazione: (2024)

Theoretical characterisation of the Gauss-Newton conditioning in Neural Networks
di: Zhao, Jim, et al.
Pubblicazione: (2024)

Cubic regularized subspace Newton for non-convex optimization
di: Zhao, Jim, et al.
Pubblicazione: (2024)

Characterizing Overfitting in Kernel Ridgeless Regression Through the Eigenspectrum
di: Cheng, Tin Sum, et al.
Pubblicazione: (2024)

Optimizer choice matters for the emergence of Neural Collapse
di: Zhao, Jim, et al.
Pubblicazione: (2026)

The informativeness of the gradient revisited
di: Takhanov, Rustem
Pubblicazione: (2025)

Initial Guessing Bias: How Untrained Networks Favor Some Classes
di: Francazi, Emanuele, et al.
Pubblicazione: (2023)

Efficient and Unbiased Sampling of Boltzmann Distributions via Consistency Models
di: Zhang, Fengzhe, et al.
Pubblicazione: (2024)

Generator Identification for Linear SDEs with Additive and Multiplicative Noise
di: Wang, Yuanyuan, et al.
Pubblicazione: (2023)

Unbiased Compression Saves Communication in Distributed Optimization: When and How Much?
di: He, Yutong, et al.
Pubblicazione: (2023)

Optimization Guarantees for Square-Root Natural-Gradient Variational Inference
di: Kumar, Navish, et al.
Pubblicazione: (2025)

Gradient Scalability and Taylor Surrogation of Quantum Cost Landscapes
di: Meyer, Sabri, et al.
Pubblicazione: (2025)

Multi-layer random features and the approximation power of neural networks
di: Takhanov, Rustem
Pubblicazione: (2024)

Small Noise Perturbations in Multidimensional Case
di: Pilipenko, Andrey, et al.
Pubblicazione: (2021)

Noise-corrected GRPO: From Noisy Rewards to Unbiased Gradients
di: Mansouri, Omar El, et al.
Pubblicazione: (2025)

Learning Unbiased Permutations via Flow Matching
di: Min, Yimeng, et al.
Pubblicazione: (2026)

Regret-Optimal Federated Transfer Learning for Kernel Regression with Applications in American Option Pricing
di: Yang, Xuwei, et al.
Pubblicazione: (2023)

StoSignSGD: Unbiased Structural Stochasticity Fixes SignSGD for Training Large Language Models
di: Yu, Dingzhi, et al.
Pubblicazione: (2026)

Learning Noise-Resilient and Transferable Graph-Text Alignment via Dynamic Quality Assessment
di: Liu, Yuhang, et al.
Pubblicazione: (2025)

Efficient and Unbiased Sampling from Boltzmann Distributions via Variance-Tuned Diffusion Models
di: Zhang, Fengzhe, et al.
Pubblicazione: (2025)

Where You Place the Norm Matters: From Prejudiced to Neutral Initializations
di: Francazi, Emanuele, et al.
Pubblicazione: (2025)

Noise-Resilient Symbolic Regression with Dynamic Gating Reinforcement Learning
di: Sun, Chenglu, et al.
Pubblicazione: (2025)

When Bias Meets Trainability: Connecting Theories of Initialization
di: Bassi, Alberto, et al.
Pubblicazione: (2025)

Byzantine-Resilient Federated Learning via Distributed Optimization
di: Xia, Yufei, et al.
Pubblicazione: (2025)