VQKV: High-Fidelity and High-Ratio Cache Compression via Vector-Quantization
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912970934583296 |
|---|---|
| author | Wang, Yixuan Shi, Qingyu Zhou, Jiayu Liu, Dianbo He, Ziwei Lin, Zhouhan |
| author_facet | Wang, Yixuan Shi, Qingyu Zhou, Jiayu Liu, Dianbo He, Ziwei Lin, Zhouhan |
| contents | The growing context length of Large Language Models (LLMs) enlarges the Key-Value (KV) cache, limiting deployment in resource-limited environments. Prior training-free approaches for KV cache compression typically rely on low-rank approximation or scalar quantization, which fail to simultaneously achieve high compression ratios and high reconstruction fidelity. We propose VQKV, a novel, training-free method introducing vector quantization (VQ) to obtain highly compressed KV representations while preserving high model fidelity, allowing for the representation of thousands of floating-point values with just a few integer indices. As a result, VQKV achieves an 82.8\% compression ratio on LLaMA3.1-8B while retaining 98.6\% of the baseline performance on LongBench and enabling 4.3x longer generation length on the same memory footprint. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_16435 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | VQKV: High-Fidelity and High-Ratio Cache Compression via Vector-Quantization Wang, Yixuan Shi, Qingyu Zhou, Jiayu Liu, Dianbo He, Ziwei Lin, Zhouhan Computation and Language The growing context length of Large Language Models (LLMs) enlarges the Key-Value (KV) cache, limiting deployment in resource-limited environments. Prior training-free approaches for KV cache compression typically rely on low-rank approximation or scalar quantization, which fail to simultaneously achieve high compression ratios and high reconstruction fidelity. We propose VQKV, a novel, training-free method introducing vector quantization (VQ) to obtain highly compressed KV representations while preserving high model fidelity, allowing for the representation of thousands of floating-point values with just a few integer indices. As a result, VQKV achieves an 82.8\% compression ratio on LLaMA3.1-8B while retaining 98.6\% of the baseline performance on LongBench and enabling 4.3x longer generation length on the same memory footprint. |
| title | VQKV: High-Fidelity and High-Ratio Cache Compression via Vector-Quantization |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2603.16435 |