Federated Learning Optimization: A Comparative Study of Data and Model Exchange Strategies in Dynamic Networks
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916288827228160 |
|---|---|
| author | Luqman, Alka Brandon, Yeow Wei Liang Chattopadhyay, Anupam |
| author_facet | Luqman, Alka Brandon, Yeow Wei Liang Chattopadhyay, Anupam |
| contents | The promise and proliferation of large-scale dynamic federated learning gives rise to a prominent open question - is it prudent to share data or model across nodes, if efficiency of transmission and fast knowledge transfer are the prime objectives. This work investigates exactly that. Specifically, we study the choices of exchanging raw data, synthetic data, or (partial) model updates among devices. The implications of these strategies in the context of foundational models are also examined in detail. Accordingly, we obtain key insights about optimal data and model exchange mechanisms considering various environments with different data distributions and dynamic device and network connections. Across various scenarios that we considered, time-limited knowledge transfer efficiency can differ by up to 9.08\%, thus highlighting the importance of this work. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_10798 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Federated Learning Optimization: A Comparative Study of Data and Model Exchange Strategies in Dynamic Networks Luqman, Alka Brandon, Yeow Wei Liang Chattopadhyay, Anupam Machine Learning Distributed, Parallel, and Cluster Computing The promise and proliferation of large-scale dynamic federated learning gives rise to a prominent open question - is it prudent to share data or model across nodes, if efficiency of transmission and fast knowledge transfer are the prime objectives. This work investigates exactly that. Specifically, we study the choices of exchanging raw data, synthetic data, or (partial) model updates among devices. The implications of these strategies in the context of foundational models are also examined in detail. Accordingly, we obtain key insights about optimal data and model exchange mechanisms considering various environments with different data distributions and dynamic device and network connections. Across various scenarios that we considered, time-limited knowledge transfer efficiency can differ by up to 9.08\%, thus highlighting the importance of this work. |
| title | Federated Learning Optimization: A Comparative Study of Data and Model Exchange Strategies in Dynamic Networks |
| topic | Machine Learning Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2406.10798 |