DASH-KV: Accelerating Long-Context LLM Inference via Asymmetric KV Cache Hashing
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910190338572288 |
|---|---|
| author | Guo, Jinyu Zhang, Zhihan Xie, Jiehui Iqbal, Md. Tamim Han, Dongshen Lee, Lik-Hang Bae, Sung-Ho Zou, Jie Yang, Yang Zhang, Chaoning |
| author_facet | Guo, Jinyu Zhang, Zhihan Xie, Jiehui Iqbal, Md. Tamim Han, Dongshen Lee, Lik-Hang Bae, Sung-Ho Zou, Jie Yang, Yang Zhang, Chaoning |
| contents | The quadratic computational complexity of the standard attention mechanism constitutes a fundamental bottleneck for large language models in long-context inference. While existing KV cache compression methods alleviate memory pressure, they often sacrifice generation quality and fail to address the high overhead of floating-point arithmetic. This paper introduces DASH-KV, an innovative acceleration framework that reformulates attention as approximate nearest-neighbor search via asymmetric deep hashing. Under this paradigm, we design an asymmetric encoding architecture that differentially maps queries and keys to account for their distinctions in precision and reuse characteristics. To balance efficiency and accuracy, we further introduce a dynamic mixed-precision mechanism that adaptively retains full-precision computation for critical tokens. Extensive experiments on LongBench demonstrate that DASH-KV significantly outperforms state-of-the-art baseline methods while matching the performance of full attention, all while reducing inference complexity from O(N^2) to linear O(N). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_19351 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | DASH-KV: Accelerating Long-Context LLM Inference via Asymmetric KV Cache Hashing Guo, Jinyu Zhang, Zhihan Xie, Jiehui Iqbal, Md. Tamim Han, Dongshen Lee, Lik-Hang Bae, Sung-Ho Zou, Jie Yang, Yang Zhang, Chaoning Computation and Language The quadratic computational complexity of the standard attention mechanism constitutes a fundamental bottleneck for large language models in long-context inference. While existing KV cache compression methods alleviate memory pressure, they often sacrifice generation quality and fail to address the high overhead of floating-point arithmetic. This paper introduces DASH-KV, an innovative acceleration framework that reformulates attention as approximate nearest-neighbor search via asymmetric deep hashing. Under this paradigm, we design an asymmetric encoding architecture that differentially maps queries and keys to account for their distinctions in precision and reuse characteristics. To balance efficiency and accuracy, we further introduce a dynamic mixed-precision mechanism that adaptively retains full-precision computation for critical tokens. Extensive experiments on LongBench demonstrate that DASH-KV significantly outperforms state-of-the-art baseline methods while matching the performance of full attention, all while reducing inference complexity from O(N^2) to linear O(N). |
| title | DASH-KV: Accelerating Long-Context LLM Inference via Asymmetric KV Cache Hashing |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2604.19351 |