Cover Image

Rethinking Key-Value Cache Compression Techniques for Large Language Model Serving

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Gao, Wei, Zhou, Xinyu, Sun, Peng, Zhang, Tianwei, Wen, Yonggang
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains More
by: Yue, Yuxuan, et al.
Published: (2024)

ReasonCache: Accelerating Large Reasoning Model Serving through KV Cache Sharing
by: Chen, Kaiwen, et al.
Published: (2025)

MTServe: Efficient Serving for Generative Recommendation Models with Hierarchical Caches
by: Wang, Xin, et al.
Published: (2026)

MiniCache: KV Cache Compression in Depth Dimension for Large Language Models
by: Liu, Akide, et al.
Published: (2024)

TorchGT: A Holistic System for Large-scale Graph Transformer Training
by: Zhang, Meng, et al.
Published: (2024)

EVICPRESS: Joint KV-Cache Compression and Eviction for Efficient LLM Serving
by: Feng, Shaoting, et al.
Published: (2025)

Advancing Generative Artificial Intelligence and Large Language Models for Demand Side Management with Internet of Electric Vehicles
by: Zhang, Hanwen, et al.
Published: (2025)

Enhancing Large Multimodal Models with Adaptive Sparsity and KV Cache Compression
by: Zhang, Te, et al.
Published: (2025)

dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Caching
by: Liu, Zhiyuan, et al.
Published: (2025)

GWT: Scalable Optimizer State Compression for Large Language Model Training
by: Wen, Ziqing, et al.
Published: (2025)

Get More with LESS: Synthesizing Recurrence with KV Cache Compression for Efficient LLM Inference
by: Dong, Harry, et al.
Published: (2024)

Fairness in Serving Large Language Models
by: Sheng, Ying, et al.
Published: (2023)

AdaptCache: KV Cache Native Storage Hierarchy for Low-Delay and High-Quality Language Model Serving
by: Feng, Shaoting, et al.
Published: (2025)

Large Language Model Compression with Global Rank and Sparsity Optimization
by: Zhou, Changhai, et al.
Published: (2025)

Sparse Prefix Caching for Hybrid and Recurrent LLM Serving
by: Shirokikh, Mikhail, et al.
Published: (2026)

Hierarchical Adaptive Eviction for KV Cache Management in Multimodal Language Models
by: Ma, Xindian, et al.
Published: (2026)

Singular Value Decomposition on Kronecker Adaptation for Large Language Model
by: Chong, Yee Hin, et al.
Published: (2025)

SPA-Cache: Singular Proxies for Adaptive Caching in Diffusion Language Models
by: Sun, Wenhao, et al.
Published: (2026)

Revisiting Service Level Objectives and System Level Metrics in Large Language Model Serving
by: Wang, Zhibin, et al.
Published: (2024)

Thin Keys, Full Values: Reducing KV Cache via Low-Dimensional Attention Selection
by: Yao, Hengshuai, et al.
Published: (2026)

The Pitfalls of KV Cache Compression
by: Chen, Alex, et al.
Published: (2025)

Joint Encoding of KV-Cache Blocks for Scalable LLM Serving
by: Kampeas, Joseph, et al.
Published: (2026)

Improving the Serving Performance of Multi-LoRA Large Language Models via Efficient LoRA and KV Cache Management
by: Zhang, Hang, et al.
Published: (2025)

Rethinking Interpretability in the Era of Large Language Models
by: Singh, Chandan, et al.
Published: (2024)

Knowledge-Aware Modeling with Frequency Adaptive Learning for Battery Health Prognostics
by: Pamshetti, Vijay Babu, et al.
Published: (2025)

When Does Value-Aware KV Eviction Help? A Fixed-Contract Diagnostic for Non-Monotone Cache Compression
by: Zhang, Ruijie, et al.
Published: (2026)

SmallKV: Small Model Assisted Compensation of KV Cache Compression for Efficient LLM Inference
by: Zhao, Yi, et al.
Published: (2025)

ZipMoE: Efficient On-Device MoE Serving via Lossless Compression and Cache-Affinity Scheduling
by: Yang, Yuchen, et al.
Published: (2026)

Palu: Compressing KV-Cache with Low-Rank Projection
by: Chang, Chi-Chih, et al.
Published: (2024)

CoSA: Compressed Sensing-Based Adaptation of Large Language Models
by: Wei, Songtao, et al.
Published: (2026)

Rethinking the Role of Temperature in Large Language Model Distillation
by: Luong, Hoang-Chau, et al.
Published: (2026)

Rethinking and Recomputing the Value of Machine Learning Models
by: Sayin, Burcu, et al.
Published: (2022)

Low-Precision Training of Large Language Models: Methods, Challenges, and Opportunities
by: Hao, Zhiwei, et al.
Published: (2025)

ShadowServe: Interference-Free KV Cache Fetching for Distributed Prefix Caching
by: Xiang, Xingyu, et al.
Published: (2025)

LoReC: Rethinking Large Language Models for Graph Data Analysis
by: Zhan, Hongyu, et al.
Published: (2026)

CrossQuant: A Post-Training Quantization Method with Smaller Quantization Kernel for Precise Large Language Model Compression
by: Liu, Wenyuan, et al.
Published: (2024)

Counterfactual Reasoning for Steerable Pluralistic Value Alignment of Large Language Models
by: Guo, Hanze, et al.
Published: (2025)

CORM: Cache Optimization with Recent Message for Large Language Model Inference
by: Dai, Jincheng, et al.
Published: (2024)

Improving Model Representation and Reducing KV Cache via Skip Connections with First Value Heads
by: Wu, Zhoutong, et al.
Published: (2025)

POLAR: Online Learning for LoRA Adapter Caching and Routing in Edge LLM Serving
by: Li, Shaoang, et al.
Published: (2026)