Distributed Saddle-Point Problems: Lower Bounds, Near-Optimal and Robust Algorithms
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2020
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915257344065536 |
|---|---|
| author | Beznosikov, Aleksandr Samokhin, Valentin Gasnikov, Alexander |
| author_facet | Beznosikov, Aleksandr Samokhin, Valentin Gasnikov, Alexander |
| contents | This paper focuses on the distributed optimization of stochastic saddle point problems. The first part of the paper is devoted to lower bounds for the centralized and decentralized distributed methods for smooth (strongly) convex-(strongly) concave saddle point problems, as well as the near-optimal algorithms by which these bounds are achieved. Next, we present a new federated algorithm for centralized distributed saddle-point problems - Extra Step Local SGD. The theoretical analysis of the new method is carried out for strongly convex-strongly concave and non-convex-non-concave problems. In the experimental part of the paper, we show the effectiveness of our method in practice. In particular, we train GANs in a distributed manner. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2010_13112 |
| institution | arXiv |
| publishDate | 2020 |
| record_format | arxiv |
| spellingShingle | Distributed Saddle-Point Problems: Lower Bounds, Near-Optimal and Robust Algorithms Beznosikov, Aleksandr Samokhin, Valentin Gasnikov, Alexander Machine Learning Distributed, Parallel, and Cluster Computing Optimization and Control This paper focuses on the distributed optimization of stochastic saddle point problems. The first part of the paper is devoted to lower bounds for the centralized and decentralized distributed methods for smooth (strongly) convex-(strongly) concave saddle point problems, as well as the near-optimal algorithms by which these bounds are achieved. Next, we present a new federated algorithm for centralized distributed saddle-point problems - Extra Step Local SGD. The theoretical analysis of the new method is carried out for strongly convex-strongly concave and non-convex-non-concave problems. In the experimental part of the paper, we show the effectiveness of our method in practice. In particular, we train GANs in a distributed manner. |
| title | Distributed Saddle-Point Problems: Lower Bounds, Near-Optimal and Robust Algorithms |
| topic | Machine Learning Distributed, Parallel, and Cluster Computing Optimization and Control |
| url | https://arxiv.org/abs/2010.13112 |