LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914850810101760 |
|---|---|
| author | Zheng, Yaowei Zhang, Richong Zhang, Junhao Ye, Yanhan Luo, Zheyan Feng, Zhangchi Ma, Yongqiang |
| author_facet | Zheng, Yaowei Zhang, Richong Zhang, Junhao Ye, Yanhan Luo, Zheyan Feng, Zhangchi Ma, Yongqiang |
| contents | Efficient fine-tuning is vital for adapting large language models (LLMs) to downstream tasks. However, it requires non-trivial efforts to implement these methods on different models. We present LlamaFactory, a unified framework that integrates a suite of cutting-edge efficient training methods. It provides a solution for flexibly customizing the fine-tuning of 100+ LLMs without the need for coding through the built-in web UI LlamaBoard. We empirically validate the efficiency and effectiveness of our framework on language modeling and text generation tasks. It has been released at https://github.com/hiyouga/LLaMA-Factory and received over 25,000 stars and 3,000 forks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_13372 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models Zheng, Yaowei Zhang, Richong Zhang, Junhao Ye, Yanhan Luo, Zheyan Feng, Zhangchi Ma, Yongqiang Computation and Language Artificial Intelligence Efficient fine-tuning is vital for adapting large language models (LLMs) to downstream tasks. However, it requires non-trivial efforts to implement these methods on different models. We present LlamaFactory, a unified framework that integrates a suite of cutting-edge efficient training methods. It provides a solution for flexibly customizing the fine-tuning of 100+ LLMs without the need for coding through the built-in web UI LlamaBoard. We empirically validate the efficiency and effectiveness of our framework on language modeling and text generation tasks. It has been released at https://github.com/hiyouga/LLaMA-Factory and received over 25,000 stars and 3,000 forks. |
| title | LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2403.13372 |