Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2505.07289 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866908360168701952 |
|---|---|
| author | Laborde, Stanislas Cousseau, Martin Yaacoub, Antoun Prevost, Lionel |
| author_facet | Laborde, Stanislas Cousseau, Martin Yaacoub, Antoun Prevost, Lionel |
| contents | The exponential growth in Large Language Model (LLM) deployment has intensified the need for efficient model compression techniques to reduce computational and memory costs. While pruning and quantization have shown promise, their combined potential remains largely unexplored. In this paper, we examine joint compression and how strategically combining pruning and quantization could yield superior performance-to-compression ratios compared to single-method approaches. Recognizing the challenges in accurately assessing LLM performance, we address key limitations of previous evaluation frameworks and introduce the Semantic Retention Compression Rate (SrCr), a novel metric that quantifies the trade-off between model compression and semantic preservation, facilitating the optimization of pruning-quantization configurations. Experiments demonstrate that our recommended combination achieves, on average, a 20% performance increase compared to an equivalent quantization-only model at the same theoretical compression rate. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_07289 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Semantic Retention and Extreme Compression in LLMs: Can We Have Both? Laborde, Stanislas Cousseau, Martin Yaacoub, Antoun Prevost, Lionel Computation and Language Artificial Intelligence Machine Learning 68P30 (Primary) 68T07, 68T50 (Secondary) I.2.6; I.5.1; I.2.7 The exponential growth in Large Language Model (LLM) deployment has intensified the need for efficient model compression techniques to reduce computational and memory costs. While pruning and quantization have shown promise, their combined potential remains largely unexplored. In this paper, we examine joint compression and how strategically combining pruning and quantization could yield superior performance-to-compression ratios compared to single-method approaches. Recognizing the challenges in accurately assessing LLM performance, we address key limitations of previous evaluation frameworks and introduce the Semantic Retention Compression Rate (SrCr), a novel metric that quantifies the trade-off between model compression and semantic preservation, facilitating the optimization of pruning-quantization configurations. Experiments demonstrate that our recommended combination achieves, on average, a 20% performance increase compared to an equivalent quantization-only model at the same theoretical compression rate. |
| title | Semantic Retention and Extreme Compression in LLMs: Can We Have Both? |
| topic | Computation and Language Artificial Intelligence Machine Learning 68P30 (Primary) 68T07, 68T50 (Secondary) I.2.6; I.5.1; I.2.7 |
| url | https://arxiv.org/abs/2505.07289 |