A Survey for Federated Learning Evaluations: Goals and Measures
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910378842128384 |
|---|---|
| author | Chai, Di Wang, Leye Yang, Liu Zhang, Junxue Chen, Kai Yang, Qiang |
| author_facet | Chai, Di Wang, Leye Yang, Liu Zhang, Junxue Chen, Kai Yang, Qiang |
| contents | Evaluation is a systematic approach to assessing how well a system achieves its intended purpose. Federated learning (FL) is a novel paradigm for privacy-preserving machine learning that allows multiple parties to collaboratively train models without sharing sensitive data. However, evaluating FL is challenging due to its interdisciplinary nature and diverse goals, such as utility, efficiency, and security. In this survey, we first review the major evaluation goals adopted in the existing studies and then explore the evaluation metrics used for each goal. We also introduce FedEval, an open-source platform that provides a standardized and comprehensive evaluation framework for FL algorithms in terms of their utility, efficiency, and security. Finally, we discuss several challenges and future research directions for FL evaluation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2308_11841 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | A Survey for Federated Learning Evaluations: Goals and Measures Chai, Di Wang, Leye Yang, Liu Zhang, Junxue Chen, Kai Yang, Qiang Machine Learning Cryptography and Security Distributed, Parallel, and Cluster Computing Evaluation is a systematic approach to assessing how well a system achieves its intended purpose. Federated learning (FL) is a novel paradigm for privacy-preserving machine learning that allows multiple parties to collaboratively train models without sharing sensitive data. However, evaluating FL is challenging due to its interdisciplinary nature and diverse goals, such as utility, efficiency, and security. In this survey, we first review the major evaluation goals adopted in the existing studies and then explore the evaluation metrics used for each goal. We also introduce FedEval, an open-source platform that provides a standardized and comprehensive evaluation framework for FL algorithms in terms of their utility, efficiency, and security. Finally, we discuss several challenges and future research directions for FL evaluation. |
| title | A Survey for Federated Learning Evaluations: Goals and Measures |
| topic | Machine Learning Cryptography and Security Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2308.11841 |