OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917587197100032 |
|---|---|
| author | Ye, Rui Wang, Wenhao Chai, Jingyi Li, Dihan Li, Zexi Xu, Yinda Du, Yaxin Wang, Yanfeng Chen, Siheng |
| author_facet | Ye, Rui Wang, Wenhao Chai, Jingyi Li, Dihan Li, Zexi Xu, Yinda Du, Yaxin Wang, Yanfeng Chen, Siheng |
| contents | Trained on massive publicly available data, large language models (LLMs) have demonstrated tremendous success across various fields. While more data contributes to better performance, a disconcerting reality is that high-quality public data will be exhausted in a few years. In this paper, we offer a potential next step for contemporary LLMs: collaborative and privacy-preserving LLM training on the underutilized distributed private data via federated learning (FL), where multiple data owners collaboratively train a shared model without transmitting raw data. To achieve this, we build a concise, integrated, and research-friendly framework/codebase, named OpenFedLLM. It covers federated instruction tuning for enhancing instruction-following capability, federated value alignment for aligning with human values, and 7 representative FL algorithms. Besides, OpenFedLLM supports training on diverse domains, where we cover 8 training datasets; and provides comprehensive evaluations, where we cover 30+ evaluation metrics. Through extensive experiments, we observe that all FL algorithms outperform local training on training LLMs, demonstrating a clear performance improvement across a variety of settings. Notably, in a financial benchmark, Llama2-7B fine-tuned by applying any FL algorithm can outperform GPT-4 by a significant margin while the model obtained through individual training cannot, demonstrating strong motivation for clients to participate in FL. The code is available at https://github.com/rui-ye/OpenFedLLM. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_06954 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learning Ye, Rui Wang, Wenhao Chai, Jingyi Li, Dihan Li, Zexi Xu, Yinda Du, Yaxin Wang, Yanfeng Chen, Siheng Machine Learning Computation and Language Distributed, Parallel, and Cluster Computing Multiagent Systems Trained on massive publicly available data, large language models (LLMs) have demonstrated tremendous success across various fields. While more data contributes to better performance, a disconcerting reality is that high-quality public data will be exhausted in a few years. In this paper, we offer a potential next step for contemporary LLMs: collaborative and privacy-preserving LLM training on the underutilized distributed private data via federated learning (FL), where multiple data owners collaboratively train a shared model without transmitting raw data. To achieve this, we build a concise, integrated, and research-friendly framework/codebase, named OpenFedLLM. It covers federated instruction tuning for enhancing instruction-following capability, federated value alignment for aligning with human values, and 7 representative FL algorithms. Besides, OpenFedLLM supports training on diverse domains, where we cover 8 training datasets; and provides comprehensive evaluations, where we cover 30+ evaluation metrics. Through extensive experiments, we observe that all FL algorithms outperform local training on training LLMs, demonstrating a clear performance improvement across a variety of settings. Notably, in a financial benchmark, Llama2-7B fine-tuned by applying any FL algorithm can outperform GPT-4 by a significant margin while the model obtained through individual training cannot, demonstrating strong motivation for clients to participate in FL. The code is available at https://github.com/rui-ye/OpenFedLLM. |
| title | OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learning |
| topic | Machine Learning Computation and Language Distributed, Parallel, and Cluster Computing Multiagent Systems |
| url | https://arxiv.org/abs/2402.06954 |