Rethinking the Representation in Federated Unsupervised Learning with Non-IID Data
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910381473005568 |
|---|---|
| author | Liao, Xinting Liu, Weiming Chen, Chaochao Zhou, Pengyang Yu, Fengyuan Zhu, Huabin Yao, Binhui Wang, Tao Zheng, Xiaolin Tan, Yanchao |
| author_facet | Liao, Xinting Liu, Weiming Chen, Chaochao Zhou, Pengyang Yu, Fengyuan Zhu, Huabin Yao, Binhui Wang, Tao Zheng, Xiaolin Tan, Yanchao |
| contents | Federated learning achieves effective performance in modeling decentralized data. In practice, client data are not well-labeled, which makes it potential for federated unsupervised learning (FUSL) with non-IID data. However, the performance of existing FUSL methods suffers from insufficient representations, i.e., (1) representation collapse entanglement among local and global models, and (2) inconsistent representation spaces among local models. The former indicates that representation collapse in local model will subsequently impact the global model and other local models. The latter means that clients model data representation with inconsistent parameters due to the deficiency of supervision signals. In this work, we propose FedU2 which enhances generating uniform and unified representation in FUSL with non-IID data. Specifically, FedU2 consists of flexible uniform regularizer (FUR) and efficient unified aggregator (EUA). FUR in each client avoids representation collapse via dispersing samples uniformly, and EUA in server promotes unified representation by constraining consistent client model updating. To extensively validate the performance of FedU2, we conduct both cross-device and cross-silo evaluation experiments on two benchmark datasets, i.e., CIFAR10 and CIFAR100. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_16398 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Rethinking the Representation in Federated Unsupervised Learning with Non-IID Data Liao, Xinting Liu, Weiming Chen, Chaochao Zhou, Pengyang Yu, Fengyuan Zhu, Huabin Yao, Binhui Wang, Tao Zheng, Xiaolin Tan, Yanchao Machine Learning Artificial Intelligence Federated learning achieves effective performance in modeling decentralized data. In practice, client data are not well-labeled, which makes it potential for federated unsupervised learning (FUSL) with non-IID data. However, the performance of existing FUSL methods suffers from insufficient representations, i.e., (1) representation collapse entanglement among local and global models, and (2) inconsistent representation spaces among local models. The former indicates that representation collapse in local model will subsequently impact the global model and other local models. The latter means that clients model data representation with inconsistent parameters due to the deficiency of supervision signals. In this work, we propose FedU2 which enhances generating uniform and unified representation in FUSL with non-IID data. Specifically, FedU2 consists of flexible uniform regularizer (FUR) and efficient unified aggregator (EUA). FUR in each client avoids representation collapse via dispersing samples uniformly, and EUA in server promotes unified representation by constraining consistent client model updating. To extensively validate the performance of FedU2, we conduct both cross-device and cross-silo evaluation experiments on two benchmark datasets, i.e., CIFAR10 and CIFAR100. |
| title | Rethinking the Representation in Federated Unsupervised Learning with Non-IID Data |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2403.16398 |