Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2405.00017 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916230089146368 |
|---|---|
| author | Leconte, Louis Jonckheere, Matthieu Samsonov, Sergey Moulines, Eric |
| author_facet | Leconte, Louis Jonckheere, Matthieu Samsonov, Sergey Moulines, Eric |
| contents | We study asynchronous federated learning mechanisms with nodes having potentially different computational speeds. In such an environment, each node is allowed to work on models with potential delays and contribute to updates to the central server at its own pace. Existing analyses of such algorithms typically depend on intractable quantities such as the maximum node delay and do not consider the underlying queuing dynamics of the system. In this paper, we propose a non-uniform sampling scheme for the central server that allows for lower delays with better complexity, taking into account the closed Jackson network structure of the associated computational graph. Our experiments clearly show a significant improvement of our method over current state-of-the-art asynchronous algorithms on an image classification problem. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_00017 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Queuing dynamics of asynchronous Federated Learning Leconte, Louis Jonckheere, Matthieu Samsonov, Sergey Moulines, Eric Distributed, Parallel, and Cluster Computing Machine Learning We study asynchronous federated learning mechanisms with nodes having potentially different computational speeds. In such an environment, each node is allowed to work on models with potential delays and contribute to updates to the central server at its own pace. Existing analyses of such algorithms typically depend on intractable quantities such as the maximum node delay and do not consider the underlying queuing dynamics of the system. In this paper, we propose a non-uniform sampling scheme for the central server that allows for lower delays with better complexity, taking into account the closed Jackson network structure of the associated computational graph. Our experiments clearly show a significant improvement of our method over current state-of-the-art asynchronous algorithms on an image classification problem. |
| title | Queuing dynamics of asynchronous Federated Learning |
| topic | Distributed, Parallel, and Cluster Computing Machine Learning |
| url | https://arxiv.org/abs/2405.00017 |