ClusComp: A Simple Paradigm for Model Compression and Efficient Finetuning
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912407942594560 |
|---|---|
| author | Liao, Baohao Herold, Christian Hashemi, Seyyed Hadi Vasilev, Stefan Khadivi, Shahram Monz, Christof |
| author_facet | Liao, Baohao Herold, Christian Hashemi, Seyyed Hadi Vasilev, Stefan Khadivi, Shahram Monz, Christof |
| contents | As large language models (LLMs) scale, model compression is crucial for edge deployment and accessibility. Weight-only quantization reduces model size but suffers from performance degradation at lower bit widths. Moreover, standard finetuning is incompatible with quantized models, and alternative methods often fall short of full finetuning. In this paper, we propose ClusComp, a simple yet effective compression paradigm that clusters weight matrices into codebooks and finetunes them block-by-block. ClusComp (1) achieves superior performance in 2-4 bit quantization, (2) pushes compression to 1-bit while outperforming ultra-low-bit methods with minimal finetuning, and (3) enables efficient finetuning, even surpassing existing quantization-based approaches and rivaling full FP16 finetuning. Notably, ClusComp supports compression and finetuning of 70B LLMs on a single A6000-48GB GPU. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_13089 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ClusComp: A Simple Paradigm for Model Compression and Efficient Finetuning Liao, Baohao Herold, Christian Hashemi, Seyyed Hadi Vasilev, Stefan Khadivi, Shahram Monz, Christof Computation and Language Artificial Intelligence As large language models (LLMs) scale, model compression is crucial for edge deployment and accessibility. Weight-only quantization reduces model size but suffers from performance degradation at lower bit widths. Moreover, standard finetuning is incompatible with quantized models, and alternative methods often fall short of full finetuning. In this paper, we propose ClusComp, a simple yet effective compression paradigm that clusters weight matrices into codebooks and finetunes them block-by-block. ClusComp (1) achieves superior performance in 2-4 bit quantization, (2) pushes compression to 1-bit while outperforming ultra-low-bit methods with minimal finetuning, and (3) enables efficient finetuning, even surpassing existing quantization-based approaches and rivaling full FP16 finetuning. Notably, ClusComp supports compression and finetuning of 70B LLMs on a single A6000-48GB GPU. |
| title | ClusComp: A Simple Paradigm for Model Compression and Efficient Finetuning |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2503.13089 |