UNIDEAL: Curriculum Knowledge Distillation Federated Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917626476756992 |
|---|---|
| author | Yang, Yuwen Liu, Chang Cai, Xun Huang, Suizhi Lu, Hongtao Ding, Yue |
| author_facet | Yang, Yuwen Liu, Chang Cai, Xun Huang, Suizhi Lu, Hongtao Ding, Yue |
| contents | Federated Learning (FL) has emerged as a promising approach to enable collaborative learning among multiple clients while preserving data privacy. However, cross-domain FL tasks, where clients possess data from different domains or distributions, remain a challenging problem due to the inherent heterogeneity. In this paper, we present UNIDEAL, a novel FL algorithm specifically designed to tackle the challenges of cross-domain scenarios and heterogeneous model architectures. The proposed method introduces Adjustable Teacher-Student Mutual Evaluation Curriculum Learning, which significantly enhances the effectiveness of knowledge distillation in FL settings. We conduct extensive experiments on various datasets, comparing UNIDEAL with state-of-the-art baselines. Our results demonstrate that UNIDEAL achieves superior performance in terms of both model accuracy and communication efficiency. Additionally, we provide a convergence analysis of the algorithm, showing a convergence rate of O(1/T) under non-convex conditions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_08961 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | UNIDEAL: Curriculum Knowledge Distillation Federated Learning Yang, Yuwen Liu, Chang Cai, Xun Huang, Suizhi Lu, Hongtao Ding, Yue Machine Learning Cryptography and Security Distributed, Parallel, and Cluster Computing Federated Learning (FL) has emerged as a promising approach to enable collaborative learning among multiple clients while preserving data privacy. However, cross-domain FL tasks, where clients possess data from different domains or distributions, remain a challenging problem due to the inherent heterogeneity. In this paper, we present UNIDEAL, a novel FL algorithm specifically designed to tackle the challenges of cross-domain scenarios and heterogeneous model architectures. The proposed method introduces Adjustable Teacher-Student Mutual Evaluation Curriculum Learning, which significantly enhances the effectiveness of knowledge distillation in FL settings. We conduct extensive experiments on various datasets, comparing UNIDEAL with state-of-the-art baselines. Our results demonstrate that UNIDEAL achieves superior performance in terms of both model accuracy and communication efficiency. Additionally, we provide a convergence analysis of the algorithm, showing a convergence rate of O(1/T) under non-convex conditions. |
| title | UNIDEAL: Curriculum Knowledge Distillation Federated Learning |
| topic | Machine Learning Cryptography and Security Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2309.08961 |