Federated Fine-Tuning of LLMs on the Very Edge: The Good, the Bad, the Ugly
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914779950481408 |
|---|---|
| author | Woisetschläger, Herbert Isenko, Alexander Wang, Shiqiang Mayer, Ruben Jacobsen, Hans-Arno |
| author_facet | Woisetschläger, Herbert Isenko, Alexander Wang, Shiqiang Mayer, Ruben Jacobsen, Hans-Arno |
| contents | Large Language Models (LLM) and foundation models are popular as they offer new opportunities for individuals and businesses to improve natural language processing, interact with data, and retrieve information faster. However, training or fine-tuning LLMs requires a vast amount of data, which can be challenging to access due to legal or technical restrictions and may require private computing resources. Federated Learning (FL) is a solution designed to overcome these challenges and expand data access for deep learning applications.
This paper takes a hardware-centric approach to explore how LLMs can be brought to modern edge computing systems. Our study fine-tunes the FLAN-T5 model family, ranging from 80M to 3B parameters, using FL for a text summarization task. We provide a micro-level hardware benchmark, compare the model FLOP utilization to a state-of-the-art data center GPU, and study the network utilization in realistic conditions. Our contribution is twofold: First, we evaluate the current capabilities of edge computing systems and their potential for LLM FL workloads. Second, by comparing these systems with a data-center GPU, we demonstrate the potential for improvement and the next steps toward achieving greater computational efficiency at the edge. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_03150 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Federated Fine-Tuning of LLMs on the Very Edge: The Good, the Bad, the Ugly Woisetschläger, Herbert Isenko, Alexander Wang, Shiqiang Mayer, Ruben Jacobsen, Hans-Arno Machine Learning Distributed, Parallel, and Cluster Computing Performance I.2.11; C.2.4; C.4; D.2.8 Large Language Models (LLM) and foundation models are popular as they offer new opportunities for individuals and businesses to improve natural language processing, interact with data, and retrieve information faster. However, training or fine-tuning LLMs requires a vast amount of data, which can be challenging to access due to legal or technical restrictions and may require private computing resources. Federated Learning (FL) is a solution designed to overcome these challenges and expand data access for deep learning applications. This paper takes a hardware-centric approach to explore how LLMs can be brought to modern edge computing systems. Our study fine-tunes the FLAN-T5 model family, ranging from 80M to 3B parameters, using FL for a text summarization task. We provide a micro-level hardware benchmark, compare the model FLOP utilization to a state-of-the-art data center GPU, and study the network utilization in realistic conditions. Our contribution is twofold: First, we evaluate the current capabilities of edge computing systems and their potential for LLM FL workloads. Second, by comparing these systems with a data-center GPU, we demonstrate the potential for improvement and the next steps toward achieving greater computational efficiency at the edge. |
| title | Federated Fine-Tuning of LLMs on the Very Edge: The Good, the Bad, the Ugly |
| topic | Machine Learning Distributed, Parallel, and Cluster Computing Performance I.2.11; C.2.4; C.4; D.2.8 |
| url | https://arxiv.org/abs/2310.03150 |