Recent Advances in Federated Learning Driven Large Language Models: A Survey on Architecture, Performance, and Security
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912366944321536 |
|---|---|
| author | Qu, Youyang Liu, Ming Zhu, Tianqing Gao, Longxiang Yu, Shui Zhou, Wanlei |
| author_facet | Qu, Youyang Liu, Ming Zhu, Tianqing Gao, Longxiang Yu, Shui Zhou, Wanlei |
| contents | Federated Learning (FL) offers a promising paradigm for training Large Language Models (LLMs) in a decentralized manner while preserving data privacy and minimizing communication overhead. This survey examines recent advancements in FL-driven LLMs, with a particular emphasis on architectural designs, performance optimization, and security concerns, including the emerging area of machine unlearning. In this context, machine unlearning refers to the systematic removal of specific data contributions from trained models to comply with privacy regulations such as the Right to be Forgotten. We review a range of strategies enabling unlearning in federated LLMs, including perturbation-based methods, model decomposition, and incremental retraining, while evaluating their trade-offs in terms of efficiency, privacy guarantees, and model utility. Through selected case studies and empirical evaluations, we analyze how these methods perform in practical FL scenarios. This survey identifies critical research directions toward developing secure, adaptable, and high-performing federated LLM systems for real-world deployment. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_09831 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Recent Advances in Federated Learning Driven Large Language Models: A Survey on Architecture, Performance, and Security Qu, Youyang Liu, Ming Zhu, Tianqing Gao, Longxiang Yu, Shui Zhou, Wanlei Machine Learning Artificial Intelligence Computation and Language Neural and Evolutionary Computing Federated Learning (FL) offers a promising paradigm for training Large Language Models (LLMs) in a decentralized manner while preserving data privacy and minimizing communication overhead. This survey examines recent advancements in FL-driven LLMs, with a particular emphasis on architectural designs, performance optimization, and security concerns, including the emerging area of machine unlearning. In this context, machine unlearning refers to the systematic removal of specific data contributions from trained models to comply with privacy regulations such as the Right to be Forgotten. We review a range of strategies enabling unlearning in federated LLMs, including perturbation-based methods, model decomposition, and incremental retraining, while evaluating their trade-offs in terms of efficiency, privacy guarantees, and model utility. Through selected case studies and empirical evaluations, we analyze how these methods perform in practical FL scenarios. This survey identifies critical research directions toward developing secure, adaptable, and high-performing federated LLM systems for real-world deployment. |
| title | Recent Advances in Federated Learning Driven Large Language Models: A Survey on Architecture, Performance, and Security |
| topic | Machine Learning Artificial Intelligence Computation and Language Neural and Evolutionary Computing |
| url | https://arxiv.org/abs/2406.09831 |