Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2405.15877 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911326809358336 |
|---|---|
| author | Li, Yang Asante, Daniel Agyei Zhao, Changsheng Chang, Ernie Shi, Yangyang Chandra, Vikas |
| author_facet | Li, Yang Asante, Daniel Agyei Zhao, Changsheng Chang, Ernie Shi, Yangyang Chandra, Vikas |
| contents | Large language models (LLMs) significantly enhance the performance of various applications, but they are computationally intensive and energy-demanding. This makes it challenging to deploy them on devices with limited resources, such as personal computers and mobile/wearable devices, and results in substantial inference costs in resource-rich environments like cloud servers. To extend the use of LLMs, we introduce a low-rank decomposition approach to effectively compress these models, tailored to the requirements of specific applications. We observe that LLMs pretrained on general datasets contain many redundant components not needed for particular applications. Our method focuses on identifying and removing these redundant parts, retaining only the necessary elements for the target applications. Specifically, we represent the weight matrices of LLMs as a linear combination of base components. We then prune the irrelevant bases and enhance the model with new bases beneficial for specific applications. Deep compression results on the Llama 2-7b and -13B models, conducted on target applications including mathematical reasoning and code generation, show that our method significantly reduces model size while maintaining comparable accuracy to state-of-the-art low-rank compression techniques. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_15877 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Basis Selection: Low-Rank Decomposition of Pretrained Large Language Models for Target Applications Li, Yang Asante, Daniel Agyei Zhao, Changsheng Chang, Ernie Shi, Yangyang Chandra, Vikas Machine Learning Hardware Architecture Computation and Language Large language models (LLMs) significantly enhance the performance of various applications, but they are computationally intensive and energy-demanding. This makes it challenging to deploy them on devices with limited resources, such as personal computers and mobile/wearable devices, and results in substantial inference costs in resource-rich environments like cloud servers. To extend the use of LLMs, we introduce a low-rank decomposition approach to effectively compress these models, tailored to the requirements of specific applications. We observe that LLMs pretrained on general datasets contain many redundant components not needed for particular applications. Our method focuses on identifying and removing these redundant parts, retaining only the necessary elements for the target applications. Specifically, we represent the weight matrices of LLMs as a linear combination of base components. We then prune the irrelevant bases and enhance the model with new bases beneficial for specific applications. Deep compression results on the Llama 2-7b and -13B models, conducted on target applications including mathematical reasoning and code generation, show that our method significantly reduces model size while maintaining comparable accuracy to state-of-the-art low-rank compression techniques. |
| title | Basis Selection: Low-Rank Decomposition of Pretrained Large Language Models for Target Applications |
| topic | Machine Learning Hardware Architecture Computation and Language |
| url | https://arxiv.org/abs/2405.15877 |