Copertina

KV Cache Compression for Inference Efficiency in LLMs: A Review

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Liu, Yanyu, Fu, Jingying, Liu, Sixiang, Zou, Yitian, Fu, You, Zhou, Jiehan, Zhang, Shouhua
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Distributed, Parallel, and Cluster Computing
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Opt-GPTQ: An Optimized GPTQ Combining Sparse Attention and Quantization Techniques
di: Kong, Jie, et al.
Pubblicazione: (2025)

LLM-CoOpt: A Co-Design and Optimization Framework for Efficient LLM Inference on Heterogeneous Platforms
di: Kong, Jie, et al.
Pubblicazione: (2026)

PackKV: Reducing KV Cache Memory Footprint through LLM-Aware Lossy Compression
di: Jiang, Bo, et al.
Pubblicazione: (2025)

KVComp: A High-Performance, LLM-Aware, Lossy Compression Framework for KV Cache
di: Jiang, Bo, et al.
Pubblicazione: (2025)

FastCache: Optimizing Multimodal LLM Serving through Lightweight KV-Cache Compression Framework
di: Zhu, Jianian, et al.
Pubblicazione: (2025)

PolyKV: A Shared Asymmetrically-Compressed KV Cache Pool for Multi-Agent LLM Inference
di: Patel, Ishan, et al.
Pubblicazione: (2026)

RcLLM: Accelerating Generative Recommendation via Beyond-Prefix KV Caching
di: Zhao, Zhan, et al.
Pubblicazione: (2026)

CacheFlow: Efficient LLM Serving with 3D-Parallel KV Cache Restoration
di: Nian, Sean, et al.
Pubblicazione: (2026)

FairKV: Balancing Per-Head KV Cache for Fast Multi-GPU Inference
di: Zhao, Bingzhe, et al.
Pubblicazione: (2025)

ContiguousKV: Accelerating LLM Prefill with Granularity-Aligned KV Cache Management
di: Zou, Jing, et al.
Pubblicazione: (2026)

KVSwap: Disk-aware KV Cache Offloading for Long-Context On-device Inference
di: Zhang, Huawei, et al.
Pubblicazione: (2025)

Mell: Memory-Efficient Large Language Model Serving via Multi-GPU KV Cache Management
di: Qianli, Liu, et al.
Pubblicazione: (2025)

Opt4GPTQ: Co-Optimizing Memory and Computation for 4-bit GPTQ Quantized LLM Inference on Heterogeneous Platforms
di: Zhang, Yaozheng, et al.
Pubblicazione: (2025)

FDC: Fast KV Dimensionality Compression for Efficient LLM Inference
di: Zhang, Zeyu, et al.
Pubblicazione: (2024)

TokenDance: Scaling Multi-Agent LLM Serving via Collective KV Cache Sharing
di: Bian, Zhuohang, et al.
Pubblicazione: (2026)

InstCache: A Predictive Cache for LLM Serving
di: Zou, Longwei, et al.
Pubblicazione: (2024)

Adaptive KV Cache Reuse for Fast Long-Context LLM Serving
di: li, Fei, et al.
Pubblicazione: (2026)

Leyline: KV Cache Directives for Agentic Inference
di: Ma, Bole, et al.
Pubblicazione: (2026)

TraCT: Disaggregated LLM Serving with CXL Shared Memory KV Cache at Rack-Scale
di: Yoon, Dongha, et al.
Pubblicazione: (2025)

FlowKV: A Disaggregated Inference Framework with Low-Latency KV Cache Transfer and Load-Aware Scheduling
di: Li, Weiqing, et al.
Pubblicazione: (2025)

Kavier: Exploring Performance, Sustainability, and Efficiency of LLM Ecosystems under Inference through Cache-Aware Discrete-Event Simulation
di: Nicolae, Radu, et al.
Pubblicazione: (2026)

InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management
di: Lee, Wonbeom, et al.
Pubblicazione: (2024)

FourierCompress: Layer-Aware Spectral Activation Compression for Efficient and Accurate Collaborative LLM Inference
di: Ma, Jian, et al.
Pubblicazione: (2025)

Efficient Remote KV Cache Reuse with GPU-native Video Codec
di: Mi, Liang, et al.
Pubblicazione: (2026)

BanaServe: Unified KV Cache and Dynamic Module Migration for Balancing Disaggregated LLM Serving in AI Infrastructure
di: He, Yiyuan, et al.
Pubblicazione: (2025)

PiKV: KV Cache Management System for Mixture of Experts
di: Liu, Dong, et al.
Pubblicazione: (2025)

FleetOpt: Analytical Fleet Provisioning for LLM Inference with Compress-and-Route as Implementation Mechanism
di: Chen, Huamin, et al.
Pubblicazione: (2026)

Efficient LLM Inference with Activation Checkpointing and Hybrid Caching
di: Lee, Sanghyeon, et al.
Pubblicazione: (2025)

Many Hands Make Light Work: Accelerating Edge Inference via Multi-Client Collaborative Caching
di: Liang, Wenyi, et al.
Pubblicazione: (2024)

Adaptive Multi-Objective Tiered Storage Configuration for KV Cache in LLM Service
di: Zheng, Xianzhe, et al.
Pubblicazione: (2026)

PerCache: Predictive Hierarchical Cache for RAG Applications on Mobile Devices
di: Liu, Kaiwei, et al.
Pubblicazione: (2025)

HACK: Homomorphic Acceleration via Compression of the Key-Value Cache for Disaggregated LLM Inference
di: Zhang, Zeyu, et al.
Pubblicazione: (2025)

ShadowServe: Interference-Free KV Cache Fetching for Distributed Prefix Caching
di: Xiang, Xingyu, et al.
Pubblicazione: (2025)

KVPR: Efficient LLM Inference with I/O-Aware KV Cache Partial Recomputation
di: Jiang, Chaoyi, et al.
Pubblicazione: (2024)

KVServe: Service-Aware KV Cache Compression for Communication-Efficient Disaggregated LLM Serving
di: Liu, Zedong, et al.
Pubblicazione: (2026)

SkyMemory: A LEO Edge Cache for Transformer Inference Optimization and Scale Out
di: Sandholm, Thomas, et al.
Pubblicazione: (2025)

FedCache: A Knowledge Cache-driven Federated Learning Architecture for Personalized Edge Intelligence
di: Wu, Zhiyuan, et al.
Pubblicazione: (2023)

The 1/W Law: An Analytical Study of Context-Length Routing Topology and GPU Generation Gains for LLM Inference Energy Efficiency
di: Chen, Huamin, et al.
Pubblicazione: (2026)

ForkKV: Scaling Multi-LoRA Agent Serving via Copy-on-Write Disaggregated KV Cache
di: Wang, Shao, et al.
Pubblicazione: (2026)

CausalMesh: A Formally Verified Causally Consistent Distributed Cache with Support for Client Migration
di: Zhang, Haoran, et al.
Pubblicazione: (2025)