Pollen: High-throughput Federated Learning Simulation via Resource-Aware Client Placement
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929347340795904 |
|---|---|
| author | Sani, Lorenzo de Gusmão, Pedro Porto Buarque Iacob, Alex Zhao, Wanru Qiu, Xinchi Gao, Yan Fernandez-Marques, Javier Lane, Nicholas Donald |
| author_facet | Sani, Lorenzo de Gusmão, Pedro Porto Buarque Iacob, Alex Zhao, Wanru Qiu, Xinchi Gao, Yan Fernandez-Marques, Javier Lane, Nicholas Donald |
| contents | Federated Learning (FL) is a privacy-focused machine learning paradigm that collaboratively trains models directly on edge devices. Simulation plays an essential role in FL adoption, helping develop novel aggregation and client sampling strategies. However, current simulators cannot emulate large-scale systems in a time-efficient manner, which limits their utility and casts doubts on generalizability.
This work proposes Pollen, a novel resource-aware system for speeding up simulations. Pollen addresses two limiting factors from existing simulators: (a) communication inefficiency derived from pull-based client execution and (b) inadequate load balance when using heterogeneous hardware. Pollen executes high-throughput FL simulations at scale by (a) using a push-based client placement system, (b) learning how an adaptable scheduling of clients based on hardware statistics (c) estimating the optimal number of concurrent workers per GPU. We evaluate Pollen on four representative FL tasks and show that Pollen's placement model increases GPU utilization and reduces idle time. We compare Pollen to Flower, Flute, FedScale, Parrot, and pfl and show experimental speed-ups of days or weeks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2306_17453 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Pollen: High-throughput Federated Learning Simulation via Resource-Aware Client Placement Sani, Lorenzo de Gusmão, Pedro Porto Buarque Iacob, Alex Zhao, Wanru Qiu, Xinchi Gao, Yan Fernandez-Marques, Javier Lane, Nicholas Donald Distributed, Parallel, and Cluster Computing Federated Learning (FL) is a privacy-focused machine learning paradigm that collaboratively trains models directly on edge devices. Simulation plays an essential role in FL adoption, helping develop novel aggregation and client sampling strategies. However, current simulators cannot emulate large-scale systems in a time-efficient manner, which limits their utility and casts doubts on generalizability. This work proposes Pollen, a novel resource-aware system for speeding up simulations. Pollen addresses two limiting factors from existing simulators: (a) communication inefficiency derived from pull-based client execution and (b) inadequate load balance when using heterogeneous hardware. Pollen executes high-throughput FL simulations at scale by (a) using a push-based client placement system, (b) learning how an adaptable scheduling of clients based on hardware statistics (c) estimating the optimal number of concurrent workers per GPU. We evaluate Pollen on four representative FL tasks and show that Pollen's placement model increases GPU utilization and reduces idle time. We compare Pollen to Flower, Flute, FedScale, Parrot, and pfl and show experimental speed-ups of days or weeks. |
| title | Pollen: High-throughput Federated Learning Simulation via Resource-Aware Client Placement |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2306.17453 |