Zeroth-Order Fine-Tuning of LLMs with Extreme Sparsity
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917685095301120 |
|---|---|
| author | Guo, Wentao Long, Jikai Zeng, Yimeng Liu, Zirui Yang, Xinyu Ran, Yide Gardner, Jacob R. Bastani, Osbert De Sa, Christopher Yu, Xiaodong Chen, Beidi Xu, Zhaozhuo |
| author_facet | Guo, Wentao Long, Jikai Zeng, Yimeng Liu, Zirui Yang, Xinyu Ran, Yide Gardner, Jacob R. Bastani, Osbert De Sa, Christopher Yu, Xiaodong Chen, Beidi Xu, Zhaozhuo |
| contents | Zeroth-order optimization (ZO) is a memory-efficient strategy for fine-tuning Large Language Models using only forward passes. However, the application of ZO fine-tuning in memory-constrained settings such as mobile phones and laptops is still challenging since full precision forward passes are infeasible. In this study, we address this limitation by integrating sparsity and quantization into ZO fine-tuning of LLMs. Specifically, we investigate the feasibility of fine-tuning an extremely small subset of LLM parameters using ZO. This approach allows the majority of un-tuned parameters to be quantized to accommodate the constraint of limited device memory. Our findings reveal that the pre-training process can identify a set of "sensitive parameters" that can guide the ZO fine-tuning of LLMs on downstream tasks. Our results demonstrate that fine-tuning 0.1% sensitive parameters in the LLM with ZO can outperform the full ZO fine-tuning performance, while offering wall-clock time speedup. Additionally, we show that ZO fine-tuning targeting these 0.1% sensitive parameters, combined with 4 bit quantization, enables efficient ZO fine-tuning of an Llama2-7B model on a GPU device with less than 8 GiB of memory and notably reduced latency. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_02913 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Zeroth-Order Fine-Tuning of LLMs with Extreme Sparsity Guo, Wentao Long, Jikai Zeng, Yimeng Liu, Zirui Yang, Xinyu Ran, Yide Gardner, Jacob R. Bastani, Osbert De Sa, Christopher Yu, Xiaodong Chen, Beidi Xu, Zhaozhuo Machine Learning Artificial Intelligence Zeroth-order optimization (ZO) is a memory-efficient strategy for fine-tuning Large Language Models using only forward passes. However, the application of ZO fine-tuning in memory-constrained settings such as mobile phones and laptops is still challenging since full precision forward passes are infeasible. In this study, we address this limitation by integrating sparsity and quantization into ZO fine-tuning of LLMs. Specifically, we investigate the feasibility of fine-tuning an extremely small subset of LLM parameters using ZO. This approach allows the majority of un-tuned parameters to be quantized to accommodate the constraint of limited device memory. Our findings reveal that the pre-training process can identify a set of "sensitive parameters" that can guide the ZO fine-tuning of LLMs on downstream tasks. Our results demonstrate that fine-tuning 0.1% sensitive parameters in the LLM with ZO can outperform the full ZO fine-tuning performance, while offering wall-clock time speedup. Additionally, we show that ZO fine-tuning targeting these 0.1% sensitive parameters, combined with 4 bit quantization, enables efficient ZO fine-tuning of an Llama2-7B model on a GPU device with less than 8 GiB of memory and notably reduced latency. |
| title | Zeroth-Order Fine-Tuning of LLMs with Extreme Sparsity |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2406.02913 |