Selective KV-Cache Sharing to Mitigate Timing Side-Channels in LLM Inference
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915787676057600 |
|---|---|
| author | Chu, Kexin Lin, Zecheng Xiang, Dawei Shen, Zixu Su, Jianchang Chu, Cheng Yang, Yiwei Zhang, Wenhui Wu, Wenfei Zhang, Wei |
| author_facet | Chu, Kexin Lin, Zecheng Xiang, Dawei Shen, Zixu Su, Jianchang Chu, Cheng Yang, Yiwei Zhang, Wenhui Wu, Wenfei Zhang, Wei |
| contents | Global KV-cache sharing is an effective optimization for accelerating large language model (LLM) inference, yet it introduces an API-visible timing side channel that lets adversaries infer sensitive user inputs from shared entries, leading to cross-tenant privacy risks. To address this problem, we introduce SafeKV (Secure and Flexible KV-cache Sharing), a system-level co-design of privacy enforcement and KV-cache management. SafeKV integrates lightweight detection and isolation directly into the serving runtime to eliminate cross-tenant reuse of sensitive KV-cache blocks under our threat model, while recovering most of the performance benefits of global sharing. Our key contributions are: (1) a three-tier asynchronous detection pipeline that decouples privacy classification from inference and supports streaming workloads, (2) a unified radix-tree-based memory manager with path compression and sensitivity-aware eviction for scalable selective isolation, and (3) an RDR-guided (Reuse Diversity Ratio) runtime safeguard that detects and bounds residual leakage. On large LLM backends, SafeKV reduces the time-to-first-token (TTFT) overhead compared to full isolation by up to 40.58% and raises throughput by up to 2.66x. Overall, SafeKV restores the efficiency of KV reuse while enforcing strong, practical privacy for multi-tenant LLM inference. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_08438 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Selective KV-Cache Sharing to Mitigate Timing Side-Channels in LLM Inference Chu, Kexin Lin, Zecheng Xiang, Dawei Shen, Zixu Su, Jianchang Chu, Cheng Yang, Yiwei Zhang, Wenhui Wu, Wenfei Zhang, Wei Cryptography and Security Machine Learning Operating Systems Global KV-cache sharing is an effective optimization for accelerating large language model (LLM) inference, yet it introduces an API-visible timing side channel that lets adversaries infer sensitive user inputs from shared entries, leading to cross-tenant privacy risks. To address this problem, we introduce SafeKV (Secure and Flexible KV-cache Sharing), a system-level co-design of privacy enforcement and KV-cache management. SafeKV integrates lightweight detection and isolation directly into the serving runtime to eliminate cross-tenant reuse of sensitive KV-cache blocks under our threat model, while recovering most of the performance benefits of global sharing. Our key contributions are: (1) a three-tier asynchronous detection pipeline that decouples privacy classification from inference and supports streaming workloads, (2) a unified radix-tree-based memory manager with path compression and sensitivity-aware eviction for scalable selective isolation, and (3) an RDR-guided (Reuse Diversity Ratio) runtime safeguard that detects and bounds residual leakage. On large LLM backends, SafeKV reduces the time-to-first-token (TTFT) overhead compared to full isolation by up to 40.58% and raises throughput by up to 2.66x. Overall, SafeKV restores the efficiency of KV reuse while enforcing strong, practical privacy for multi-tenant LLM inference. |
| title | Selective KV-Cache Sharing to Mitigate Timing Side-Channels in LLM Inference |
| topic | Cryptography and Security Machine Learning Operating Systems |
| url | https://arxiv.org/abs/2508.08438 |