Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2509.17659 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909799581483008 |
|---|---|
| author | Yu, Zhan Liao, Lan Yuan, Deming Ho, Daniel W. C. Zhou, Ding-Xuan |
| author_facet | Yu, Zhan Liao, Lan Yuan, Deming Ho, Daniel W. C. Zhou, Ding-Xuan |
| contents | We study the distributed stochastic optimization (DSO) problem under a heavy-tailed noise condition by utilizing a multi-agent system. Despite the extensive research on DSO algorithms used to solve DSO problems under light-tailed noise conditions (such as Gaussian noise), there is a significant lack of study of DSO algorithms in the context of heavy-tailed random noise. Classical DSO approaches in a heavy-tailed setting may present poor convergence behaviors. Therefore, developing DSO methods in the context of heavy-tailed noises is of importance. This work follows this path and we consider the setting that the gradient noises associated with each agent can be heavy-tailed, potentially having unbounded variance. We propose a clipped federated stochastic mirror descent algorithm to solve the DSO problem. We rigorously present a convergence theory and show that, under appropriate rules on the stepsize and the clipping parameter associated with the local noisy gradient influenced by the heavy-tailed noise, the algorithm is able to achieve satisfactory high probability convergence. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_17659 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Distributed Stochastic Optimization under Heavy-Tailed Noise: A Federated Mirror Descent Approach with High Probability Convergence Yu, Zhan Liao, Lan Yuan, Deming Ho, Daniel W. C. Zhou, Ding-Xuan Optimization and Control We study the distributed stochastic optimization (DSO) problem under a heavy-tailed noise condition by utilizing a multi-agent system. Despite the extensive research on DSO algorithms used to solve DSO problems under light-tailed noise conditions (such as Gaussian noise), there is a significant lack of study of DSO algorithms in the context of heavy-tailed random noise. Classical DSO approaches in a heavy-tailed setting may present poor convergence behaviors. Therefore, developing DSO methods in the context of heavy-tailed noises is of importance. This work follows this path and we consider the setting that the gradient noises associated with each agent can be heavy-tailed, potentially having unbounded variance. We propose a clipped federated stochastic mirror descent algorithm to solve the DSO problem. We rigorously present a convergence theory and show that, under appropriate rules on the stepsize and the clipping parameter associated with the local noisy gradient influenced by the heavy-tailed noise, the algorithm is able to achieve satisfactory high probability convergence. |
| title | Distributed Stochastic Optimization under Heavy-Tailed Noise: A Federated Mirror Descent Approach with High Probability Convergence |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2509.17659 |