FedStein: Enhancing Multi-Domain Federated Learning Through James-Stein Estimator
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913531998240768 |
|---|---|
| author | Gupta, Sunny Jangid, Nikita Sethi, Amit |
| author_facet | Gupta, Sunny Jangid, Nikita Sethi, Amit |
| contents | Federated Learning (FL) facilitates data privacy by enabling collaborative in-situ training across decentralized clients. Despite its inherent advantages, FL faces significant challenges of performance and convergence when dealing with data that is not independently and identically distributed (non-i.i.d.). While previous research has primarily addressed the issue of skewed label distribution across clients, this study focuses on the less explored challenge of multi-domain FL, where client data originates from distinct domains with varying feature distributions. We introduce a novel method designed to address these challenges FedStein: Enhancing Multi-Domain Federated Learning Through the James-Stein Estimator. FedStein uniquely shares only the James-Stein (JS) estimates of batch normalization (BN) statistics across clients, while maintaining local BN parameters. The non-BN layer parameters are exchanged via standard FL techniques. Extensive experiments conducted across three datasets and multiple models demonstrate that FedStein surpasses existing methods such as FedAvg and FedBN, with accuracy improvements exceeding 14% in certain domains leading to enhanced domain generalization. The code is available at https://github.com/sunnyinAI/FedStein |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_03499 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | FedStein: Enhancing Multi-Domain Federated Learning Through James-Stein Estimator Gupta, Sunny Jangid, Nikita Sethi, Amit Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition Distributed, Parallel, and Cluster Computing I.2.6; C.1.4; D.1.3; I.5.1; H.3.4; I.2.10; I.4.0; I.4.1; I.4.2; I.4.6; I.4.7; I.4.8; I.4.9; I.4.10; I.5.1; I.5.2; I.5.4; J.2; I.2.11; I.2.10 Federated Learning (FL) facilitates data privacy by enabling collaborative in-situ training across decentralized clients. Despite its inherent advantages, FL faces significant challenges of performance and convergence when dealing with data that is not independently and identically distributed (non-i.i.d.). While previous research has primarily addressed the issue of skewed label distribution across clients, this study focuses on the less explored challenge of multi-domain FL, where client data originates from distinct domains with varying feature distributions. We introduce a novel method designed to address these challenges FedStein: Enhancing Multi-Domain Federated Learning Through the James-Stein Estimator. FedStein uniquely shares only the James-Stein (JS) estimates of batch normalization (BN) statistics across clients, while maintaining local BN parameters. The non-BN layer parameters are exchanged via standard FL techniques. Extensive experiments conducted across three datasets and multiple models demonstrate that FedStein surpasses existing methods such as FedAvg and FedBN, with accuracy improvements exceeding 14% in certain domains leading to enhanced domain generalization. The code is available at https://github.com/sunnyinAI/FedStein |
| title | FedStein: Enhancing Multi-Domain Federated Learning Through James-Stein Estimator |
| topic | Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition Distributed, Parallel, and Cluster Computing I.2.6; C.1.4; D.1.3; I.5.1; H.3.4; I.2.10; I.4.0; I.4.1; I.4.2; I.4.6; I.4.7; I.4.8; I.4.9; I.4.10; I.5.1; I.5.2; I.5.4; J.2; I.2.11; I.2.10 |
| url | https://arxiv.org/abs/2410.03499 |