Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2301.12195 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916330568941568 |
|---|---|
| author | Feng, Haozhe Pang, Tianyu Du, Chao Chen, Wei Yan, Shuicheng Lin, Min |
| author_facet | Feng, Haozhe Pang, Tianyu Du, Chao Chen, Wei Yan, Shuicheng Lin, Min |
| contents | Federated learning (FL) is a general principle for decentralized clients to train a server model collectively without sharing local data. FL is a promising framework with practical applications, but its standard training paradigm requires the clients to backpropagate through the model to compute gradients. Since these clients are typically edge devices and not fully trusted, executing backpropagation on them incurs computational and storage overhead as well as white-box vulnerability. In light of this, we develop backpropagation-free federated learning, dubbed BAFFLE, in which backpropagation is replaced by multiple forward processes to estimate gradients. BAFFLE is 1) memory-efficient and easily fits uploading bandwidth; 2) compatible with inference-only hardware optimization and model quantization or pruning; and 3) well-suited to trusted execution environments, because the clients in BAFFLE only execute forward propagation and return a set of scalars to the server. Empirically we use BAFFLE to train deep models from scratch or to finetune pretrained models, achieving acceptable results. Code is available in https://github.com/FengHZ/BAFFLE. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2301_12195 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | BAFFLE: A Baseline of Backpropagation-Free Federated Learning Feng, Haozhe Pang, Tianyu Du, Chao Chen, Wei Yan, Shuicheng Lin, Min Machine Learning Artificial Intelligence Cryptography and Security Distributed, Parallel, and Cluster Computing Federated learning (FL) is a general principle for decentralized clients to train a server model collectively without sharing local data. FL is a promising framework with practical applications, but its standard training paradigm requires the clients to backpropagate through the model to compute gradients. Since these clients are typically edge devices and not fully trusted, executing backpropagation on them incurs computational and storage overhead as well as white-box vulnerability. In light of this, we develop backpropagation-free federated learning, dubbed BAFFLE, in which backpropagation is replaced by multiple forward processes to estimate gradients. BAFFLE is 1) memory-efficient and easily fits uploading bandwidth; 2) compatible with inference-only hardware optimization and model quantization or pruning; and 3) well-suited to trusted execution environments, because the clients in BAFFLE only execute forward propagation and return a set of scalars to the server. Empirically we use BAFFLE to train deep models from scratch or to finetune pretrained models, achieving acceptable results. Code is available in https://github.com/FengHZ/BAFFLE. |
| title | BAFFLE: A Baseline of Backpropagation-Free Federated Learning |
| topic | Machine Learning Artificial Intelligence Cryptography and Security Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2301.12195 |