Eliminating Domain Bias for Federated Learning in Representation Space
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910748191490048 |
|---|---|
| author | Zhang, Jianqing Hua, Yang Cao, Jian Wang, Hao Song, Tao Xue, Zhengui Ma, Ruhui Guan, Haibing |
| author_facet | Zhang, Jianqing Hua, Yang Cao, Jian Wang, Hao Song, Tao Xue, Zhengui Ma, Ruhui Guan, Haibing |
| contents | Recently, federated learning (FL) is popular for its privacy-preserving and collaborative learning abilities. However, under statistically heterogeneous scenarios, we observe that biased data domains on clients cause a representation bias phenomenon and further degenerate generic representations during local training, i.e., the representation degeneration phenomenon. To address these issues, we propose a general framework Domain Bias Eliminator (DBE) for FL. Our theoretical analysis reveals that DBE can promote bi-directional knowledge transfer between server and client, as it reduces the domain discrepancy between server and client in representation space. Besides, extensive experiments on four datasets show that DBE can greatly improve existing FL methods in both generalization and personalization abilities. The DBE-equipped FL method can outperform ten state-of-the-art personalized FL methods by a large margin. Our code is public at https://github.com/TsingZ0/DBE. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_14975 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Eliminating Domain Bias for Federated Learning in Representation Space Zhang, Jianqing Hua, Yang Cao, Jian Wang, Hao Song, Tao Xue, Zhengui Ma, Ruhui Guan, Haibing Machine Learning Distributed, Parallel, and Cluster Computing Recently, federated learning (FL) is popular for its privacy-preserving and collaborative learning abilities. However, under statistically heterogeneous scenarios, we observe that biased data domains on clients cause a representation bias phenomenon and further degenerate generic representations during local training, i.e., the representation degeneration phenomenon. To address these issues, we propose a general framework Domain Bias Eliminator (DBE) for FL. Our theoretical analysis reveals that DBE can promote bi-directional knowledge transfer between server and client, as it reduces the domain discrepancy between server and client in representation space. Besides, extensive experiments on four datasets show that DBE can greatly improve existing FL methods in both generalization and personalization abilities. The DBE-equipped FL method can outperform ten state-of-the-art personalized FL methods by a large margin. Our code is public at https://github.com/TsingZ0/DBE. |
| title | Eliminating Domain Bias for Federated Learning in Representation Space |
| topic | Machine Learning Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2311.14975 |