Pruning Federated Models through Loss Landscape Analysis and Client Agreement Scoring
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913117533896704 |
|---|---|
| author | Internò, Christian Raponi, Elena Olhofer, Markus Raza, Ali Bäck, Thomas van Stein, Niki Jin, Yaochu Hammer, Barbara |
| author_facet | Internò, Christian Raponi, Elena Olhofer, Markus Raza, Ali Bäck, Thomas van Stein, Niki Jin, Yaochu Hammer, Barbara |
| contents | The practical deployment of Federated Learning (FL) on resource-constrained devices is fundamentally limited by the high cost of training large models and the instability caused by heterogeneous (non-IID) client data. Conventional pruning methods often treat data heterogeneity as a problem to be mitigated. In this work, we introduce a paradigm shift: we reframe client diversity as a feature to be harnessed. We propose AutoFLIP, a framework that begins not with training, but with a one-time federated loss exploration. During this phase, clients collaboratively build a map of the collective loss landscape, using their diverse data to reveal the problem's essential structure. This shared intelligence then guides an adaptive pruning strategy that is dynamically refined by client agreement throughout training. This approach allows AutoFLIP to identify robust and efficient sub-networks from the outset. Our extensive experiments show that AutoFLIP reduces computational overhead by an average of 52% and communication costs by over 65% while simultaneously achieving state-of-the-art accuracy in challenging non-IID settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_10271 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Pruning Federated Models through Loss Landscape Analysis and Client Agreement Scoring Internò, Christian Raponi, Elena Olhofer, Markus Raza, Ali Bäck, Thomas van Stein, Niki Jin, Yaochu Hammer, Barbara Machine Learning Artificial Intelligence Distributed, Parallel, and Cluster Computing Emerging Technologies The practical deployment of Federated Learning (FL) on resource-constrained devices is fundamentally limited by the high cost of training large models and the instability caused by heterogeneous (non-IID) client data. Conventional pruning methods often treat data heterogeneity as a problem to be mitigated. In this work, we introduce a paradigm shift: we reframe client diversity as a feature to be harnessed. We propose AutoFLIP, a framework that begins not with training, but with a one-time federated loss exploration. During this phase, clients collaboratively build a map of the collective loss landscape, using their diverse data to reveal the problem's essential structure. This shared intelligence then guides an adaptive pruning strategy that is dynamically refined by client agreement throughout training. This approach allows AutoFLIP to identify robust and efficient sub-networks from the outset. Our extensive experiments show that AutoFLIP reduces computational overhead by an average of 52% and communication costs by over 65% while simultaneously achieving state-of-the-art accuracy in challenging non-IID settings. |
| title | Pruning Federated Models through Loss Landscape Analysis and Client Agreement Scoring |
| topic | Machine Learning Artificial Intelligence Distributed, Parallel, and Cluster Computing Emerging Technologies |
| url | https://arxiv.org/abs/2405.10271 |