FedPhD: Federated Pruning with Hierarchical Learning of Diffusion Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908441649348608 |
|---|---|
| author | Long, Qianyu Wang, Qiyuan Anagnostopoulos, Christos Bi, Daning |
| author_facet | Long, Qianyu Wang, Qiyuan Anagnostopoulos, Christos Bi, Daning |
| contents | Federated Learning (FL), as a distributed learning paradigm, trains models over distributed clients' data. FL is particularly beneficial for distributed training of Diffusion Models (DMs), which are high-quality image generators that require diverse data. However, challenges such as high communication costs and data heterogeneity persist in training DMs similar to training Transformers and Convolutional Neural Networks. Limited research has addressed these issues in FL environments. To address this gap and challenges, we introduce a novel approach, FedPhD, designed to efficiently train DMs in FL environments. FedPhD leverages Hierarchical FL with homogeneity-aware model aggregation and selection policy to tackle data heterogeneity while reducing communication costs. The distributed structured pruning of FedPhD enhances computational efficiency and reduces model storage requirements in clients. Our experiments across multiple datasets demonstrate that FedPhD achieves high model performance regarding Fréchet Inception Distance (FID) scores while reducing communication costs by up to $88\%$. FedPhD outperforms baseline methods achieving at least a $34\%$ improvement in FID, while utilizing only $56\%$ of the total computation and communication resources. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_06449 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | FedPhD: Federated Pruning with Hierarchical Learning of Diffusion Models Long, Qianyu Wang, Qiyuan Anagnostopoulos, Christos Bi, Daning Machine Learning Artificial Intelligence Distributed, Parallel, and Cluster Computing 68T05, 68T07, 68Q85, 94A08 I.2.6; I.2.11; C.2.4 Federated Learning (FL), as a distributed learning paradigm, trains models over distributed clients' data. FL is particularly beneficial for distributed training of Diffusion Models (DMs), which are high-quality image generators that require diverse data. However, challenges such as high communication costs and data heterogeneity persist in training DMs similar to training Transformers and Convolutional Neural Networks. Limited research has addressed these issues in FL environments. To address this gap and challenges, we introduce a novel approach, FedPhD, designed to efficiently train DMs in FL environments. FedPhD leverages Hierarchical FL with homogeneity-aware model aggregation and selection policy to tackle data heterogeneity while reducing communication costs. The distributed structured pruning of FedPhD enhances computational efficiency and reduces model storage requirements in clients. Our experiments across multiple datasets demonstrate that FedPhD achieves high model performance regarding Fréchet Inception Distance (FID) scores while reducing communication costs by up to $88\%$. FedPhD outperforms baseline methods achieving at least a $34\%$ improvement in FID, while utilizing only $56\%$ of the total computation and communication resources. |
| title | FedPhD: Federated Pruning with Hierarchical Learning of Diffusion Models |
| topic | Machine Learning Artificial Intelligence Distributed, Parallel, and Cluster Computing 68T05, 68T07, 68Q85, 94A08 I.2.6; I.2.11; C.2.4 |
| url | https://arxiv.org/abs/2507.06449 |