Scalable and Resource-Efficient Second-Order Federated Learning via Over-the-Air Aggregation
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929675852316672 |
|---|---|
| author | Ghalkha, Abdulmomen Issaid, Chaouki Ben Bennis, Mehdi |
| author_facet | Ghalkha, Abdulmomen Issaid, Chaouki Ben Bennis, Mehdi |
| contents | Second-order federated learning (FL) algorithms offer faster convergence than their first-order counterparts by leveraging curvature information. However, they are hindered by high computational and storage costs, particularly for large-scale models. Furthermore, the communication overhead associated with large models and digital transmission exacerbates these challenges, causing communication bottlenecks. In this work, we propose a scalable second-order FL algorithm using a sparse Hessian estimate and leveraging over-the-air aggregation, making it feasible for larger models. Our simulation results demonstrate more than $67\%$ of communication resources and energy savings compared to other first and second-order baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_07662 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Scalable and Resource-Efficient Second-Order Federated Learning via Over-the-Air Aggregation Ghalkha, Abdulmomen Issaid, Chaouki Ben Bennis, Mehdi Machine Learning Second-order federated learning (FL) algorithms offer faster convergence than their first-order counterparts by leveraging curvature information. However, they are hindered by high computational and storage costs, particularly for large-scale models. Furthermore, the communication overhead associated with large models and digital transmission exacerbates these challenges, causing communication bottlenecks. In this work, we propose a scalable second-order FL algorithm using a sparse Hessian estimate and leveraging over-the-air aggregation, making it feasible for larger models. Our simulation results demonstrate more than $67\%$ of communication resources and energy savings compared to other first and second-order baselines. |
| title | Scalable and Resource-Efficient Second-Order Federated Learning via Over-the-Air Aggregation |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2410.07662 |