Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2409.15723 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917104349872128 |
|---|---|
| author | Yao, Yuhang Zhang, Jianyi Wu, Junda Huang, Chengkai Xia, Yu Yu, Tong Zhang, Ruiyi Kim, Sungchul Rossi, Ryan Li, Ang Yao, Lina McAuley, Julian Chen, Yiran Joe-Wong, Carlee |
| author_facet | Yao, Yuhang Zhang, Jianyi Wu, Junda Huang, Chengkai Xia, Yu Yu, Tong Zhang, Ruiyi Kim, Sungchul Rossi, Ryan Li, Ang Yao, Lina McAuley, Julian Chen, Yiran Joe-Wong, Carlee |
| contents | Large language models are rapidly gaining popularity and have been widely adopted in real-world applications. While the quality of training data is essential, privacy concerns arise during data collection. Federated learning offers a solution by allowing multiple clients to collaboratively train LLMs without sharing local data. However, FL introduces new challenges, such as model convergence issues due to heterogeneous data and high communication costs. A comprehensive study is required to address these challenges and guide future research. This paper surveys Federated learning for LLMs (FedLLM), highlighting recent advances and future directions. We focus on two key aspects: fine-tuning and prompt learning in a federated setting, discussing existing work and associated research challenges. We finally propose potential directions for federated LLMs, including pre-training, federated agents, and LLMs for federated learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_15723 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Federated Large Language Models: Current Progress and Future Directions Yao, Yuhang Zhang, Jianyi Wu, Junda Huang, Chengkai Xia, Yu Yu, Tong Zhang, Ruiyi Kim, Sungchul Rossi, Ryan Li, Ang Yao, Lina McAuley, Julian Chen, Yiran Joe-Wong, Carlee Machine Learning Computation and Language Large language models are rapidly gaining popularity and have been widely adopted in real-world applications. While the quality of training data is essential, privacy concerns arise during data collection. Federated learning offers a solution by allowing multiple clients to collaboratively train LLMs without sharing local data. However, FL introduces new challenges, such as model convergence issues due to heterogeneous data and high communication costs. A comprehensive study is required to address these challenges and guide future research. This paper surveys Federated learning for LLMs (FedLLM), highlighting recent advances and future directions. We focus on two key aspects: fine-tuning and prompt learning in a federated setting, discussing existing work and associated research challenges. We finally propose potential directions for federated LLMs, including pre-training, federated agents, and LLMs for federated learning. |
| title | Federated Large Language Models: Current Progress and Future Directions |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2409.15723 |