Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2501.10347 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915531386257408 |
|---|---|
| author | Feng, Chao Kohler, Nicolas Fazli Wang, Zhi Niu, Weijie Celdran, Alberto Huertas Bovet, Gerome Stiller, Burkhard |
| author_facet | Feng, Chao Kohler, Nicolas Fazli Wang, Zhi Niu, Weijie Celdran, Alberto Huertas Bovet, Gerome Stiller, Burkhard |
| contents | The integration of Federated Learning (FL) and Multi-Task Learning (MTL) has been explored to address client heterogeneity, with Federated Multi-Task Learning (FMTL) treating each client as a distinct task. However, most existing research focuses on data heterogeneity (e.g., addressing non-IID data) rather than task heterogeneity, where clients solve fundamentally different tasks. Additionally, much of the work relies on centralized settings with a server managing the federation, leaving the more challenging domain of decentralized FMTL largely unexplored. Thus, this work bridges this gap by proposing ColNet, a framework designed for heterogeneous tasks in decentralized federated environments.
ColNet partitions models into a backbone and task-specific heads, and uses adaptive clustering based on model and data sensitivity to form task-coherent client groups. Backbones are averaged within groups, and group leaders perform hyper-conflict-averse cross-group aggregation. Across datasets and federations, ColNet outperforms competing schemes under label and task heterogeneity and shows robustness to poisoning attacks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_10347 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ColNet: Collaborative Optimization in Decentralized Federated Multi-task Learning Systems Feng, Chao Kohler, Nicolas Fazli Wang, Zhi Niu, Weijie Celdran, Alberto Huertas Bovet, Gerome Stiller, Burkhard Machine Learning The integration of Federated Learning (FL) and Multi-Task Learning (MTL) has been explored to address client heterogeneity, with Federated Multi-Task Learning (FMTL) treating each client as a distinct task. However, most existing research focuses on data heterogeneity (e.g., addressing non-IID data) rather than task heterogeneity, where clients solve fundamentally different tasks. Additionally, much of the work relies on centralized settings with a server managing the federation, leaving the more challenging domain of decentralized FMTL largely unexplored. Thus, this work bridges this gap by proposing ColNet, a framework designed for heterogeneous tasks in decentralized federated environments. ColNet partitions models into a backbone and task-specific heads, and uses adaptive clustering based on model and data sensitivity to form task-coherent client groups. Backbones are averaged within groups, and group leaders perform hyper-conflict-averse cross-group aggregation. Across datasets and federations, ColNet outperforms competing schemes under label and task heterogeneity and shows robustness to poisoning attacks. |
| title | ColNet: Collaborative Optimization in Decentralized Federated Multi-task Learning Systems |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2501.10347 |