Semantic Integrity Matters: Benchmarking and Preserving High-Density Reasoning in KV Cache Compression
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914556002959360 |
|---|---|
| author | Liu, Xiang Tang, Zhenheng Chen, Hong Dong, Peijie Li, Zeyu Zhou, Xiuze Li, Bo Hu, Xuming Chu, Xiaowen |
| author_facet | Liu, Xiang Tang, Zhenheng Chen, Hong Dong, Peijie Li, Zeyu Zhou, Xiuze Li, Bo Hu, Xuming Chu, Xiaowen |
| contents | While Key-Value (KV) cache compression is essential for efficient LLM inference, current evaluations disproportionately focus on sparse retrieval tasks, potentially masking the degradation of High-Density Reasoning where Chain-of-Thought (CoT) coherence is critical. We introduce KVFundaBench to systematically evaluate this gap, revealing a sharp dichotomy: while retrieval tasks remain robust, reasoning tasks exhibit severe Task-Dependent Degradation under aggressive compression due to disrupted CoT links. Extending our analysis to the DeepSeek-R1 model, we uncover that its specialized attention patterns offer unique insights into the fragility of reasoning chains. Guided by these findings -- specifically the necessity of preserving few-shot examples as indivisible Semantic Units -- we propose ShotKV. This approach explicitly separates prefill and decoding phases to prioritize semantic integrity. Empirical results demonstrate that ShotKV achieves 9%-18% accuracy improvements on long-context generation tasks and effectively generalizes to document QA, all while delivering an 11% latency reduction compared to full cache inference. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_01941 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Semantic Integrity Matters: Benchmarking and Preserving High-Density Reasoning in KV Cache Compression Liu, Xiang Tang, Zhenheng Chen, Hong Dong, Peijie Li, Zeyu Zhou, Xiuze Li, Bo Hu, Xuming Chu, Xiaowen Computation and Language Artificial Intelligence While Key-Value (KV) cache compression is essential for efficient LLM inference, current evaluations disproportionately focus on sparse retrieval tasks, potentially masking the degradation of High-Density Reasoning where Chain-of-Thought (CoT) coherence is critical. We introduce KVFundaBench to systematically evaluate this gap, revealing a sharp dichotomy: while retrieval tasks remain robust, reasoning tasks exhibit severe Task-Dependent Degradation under aggressive compression due to disrupted CoT links. Extending our analysis to the DeepSeek-R1 model, we uncover that its specialized attention patterns offer unique insights into the fragility of reasoning chains. Guided by these findings -- specifically the necessity of preserving few-shot examples as indivisible Semantic Units -- we propose ShotKV. This approach explicitly separates prefill and decoding phases to prioritize semantic integrity. Empirical results demonstrate that ShotKV achieves 9%-18% accuracy improvements on long-context generation tasks and effectively generalizes to document QA, all while delivering an 11% latency reduction compared to full cache inference. |
| title | Semantic Integrity Matters: Benchmarking and Preserving High-Density Reasoning in KV Cache Compression |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2502.01941 |