FedRE: A Representation Entanglement Framework for Model-Heterogeneous Federated Learning
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910076418129920 |
|---|---|
| author | Yao, Yuan Wang, Lixu Wu, Jiaqi Song, Jin Chen, Simin Wang, Zehua Tian, Zijian Chen, Wei Li, Huixia Li, Xiaoxiao |
| author_facet | Yao, Yuan Wang, Lixu Wu, Jiaqi Song, Jin Chen, Simin Wang, Zehua Tian, Zijian Chen, Wei Li, Huixia Li, Xiaoxiao |
| contents | Federated learning (FL) enables collaborative training across clients while preserving privacy. While most existing FL methods assume homogeneous model architectures, client heterogeneity in both data and resources makes this assumption impractical, thus motivating model-heterogeneous FL. To address this problem, we propose Federated Representation Entanglement (FedRE), a framework built upon a novel form of client knowledge termed entangled representation. Specifically, each client aggregates its local representations into a single entangled representation using normalized random weights, and then applies the same weights to integrate the corresponding one-hot label encodings into an entangled-label encoding. Both are subsequently uploaded to the server to train a global classifier. During training, each entangled representation is supervised across categories via its entangled-label encoding, while random weights are re-sampled at each round to introduce diversity, alleviating overconfidence in the global classifier and yielding smoother decision boundaries. Moreover, each client uploads a single entangled representation along with its entangled-label encoding, mitigating the risk of representation inversion attacks and reducing communication overhead. Extensive experiments demonstrate that FedRE achieves an effective trade-off among model performance, privacy protection, and communication overhead. The codes are available at https://github.com/AIResearch-Group/FedRE. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_22265 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | FedRE: A Representation Entanglement Framework for Model-Heterogeneous Federated Learning Yao, Yuan Wang, Lixu Wu, Jiaqi Song, Jin Chen, Simin Wang, Zehua Tian, Zijian Chen, Wei Li, Huixia Li, Xiaoxiao Machine Learning Federated learning (FL) enables collaborative training across clients while preserving privacy. While most existing FL methods assume homogeneous model architectures, client heterogeneity in both data and resources makes this assumption impractical, thus motivating model-heterogeneous FL. To address this problem, we propose Federated Representation Entanglement (FedRE), a framework built upon a novel form of client knowledge termed entangled representation. Specifically, each client aggregates its local representations into a single entangled representation using normalized random weights, and then applies the same weights to integrate the corresponding one-hot label encodings into an entangled-label encoding. Both are subsequently uploaded to the server to train a global classifier. During training, each entangled representation is supervised across categories via its entangled-label encoding, while random weights are re-sampled at each round to introduce diversity, alleviating overconfidence in the global classifier and yielding smoother decision boundaries. Moreover, each client uploads a single entangled representation along with its entangled-label encoding, mitigating the risk of representation inversion attacks and reducing communication overhead. Extensive experiments demonstrate that FedRE achieves an effective trade-off among model performance, privacy protection, and communication overhead. The codes are available at https://github.com/AIResearch-Group/FedRE. |
| title | FedRE: A Representation Entanglement Framework for Model-Heterogeneous Federated Learning |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2511.22265 |