Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2412.11378 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917898075766784 |
|---|---|
| author | Wang, Dannong Kim, Daniel Jin, Bo Zhao, Xingjian Fu, Tianfan Yang, Steve Liu, Xiao-Yang |
| author_facet | Wang, Dannong Kim, Daniel Jin, Bo Zhao, Xingjian Fu, Tianfan Yang, Steve Liu, Xiao-Yang |
| contents | Finetuned large language models (LLMs) have shown remarkable performance in financial tasks, such as sentiment analysis and information retrieval. Due to privacy concerns, finetuning and deploying Financial LLMs (FinLLMs) locally are crucial for institutions. However, finetuning FinLLMs poses challenges including GPU memory constraints and long input sequences. In this paper, we employ quantized low-rank adaptation (QLoRA) to finetune FinLLMs, which leverage low-rank matrix decomposition and quantization techniques to significantly reduce computational requirements while maintaining high model performance. We also employ data and pipeline parallelism to enable local finetuning using cost-effective, widely accessible GPUs. Experiments on financial datasets demonstrate that our method achieves substantial improvements in accuracy, GPU memory usage, and time efficiency, underscoring the potential of lowrank methods for scalable and resource-efficient LLM finetuning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_11378 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | FinLoRA: Finetuning Quantized Financial Large Language Models Using Low-Rank Adaptation Wang, Dannong Kim, Daniel Jin, Bo Zhao, Xingjian Fu, Tianfan Yang, Steve Liu, Xiao-Yang Machine Learning Finetuned large language models (LLMs) have shown remarkable performance in financial tasks, such as sentiment analysis and information retrieval. Due to privacy concerns, finetuning and deploying Financial LLMs (FinLLMs) locally are crucial for institutions. However, finetuning FinLLMs poses challenges including GPU memory constraints and long input sequences. In this paper, we employ quantized low-rank adaptation (QLoRA) to finetune FinLLMs, which leverage low-rank matrix decomposition and quantization techniques to significantly reduce computational requirements while maintaining high model performance. We also employ data and pipeline parallelism to enable local finetuning using cost-effective, widely accessible GPUs. Experiments on financial datasets demonstrate that our method achieves substantial improvements in accuracy, GPU memory usage, and time efficiency, underscoring the potential of lowrank methods for scalable and resource-efficient LLM finetuning. |
| title | FinLoRA: Finetuning Quantized Financial Large Language Models Using Low-Rank Adaptation |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2412.11378 |