Cover Image

CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Yuhan, Li, Hanchen, Cheng, Yihua, Ray, Siddhant, Huang, Yuyang, Zhang, Qizheng, Du, Kuntai, Yao, Jiayi, Lu, Shan, Ananthanarayanan, Ganesh, Maire, Michael, Hoffmann, Henry, Holtzman, Ari, Jiang, Junchen
Format:	Preprint
Published:	2023
Subjects:	Networking and Internet Architecture Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion
by: Yao, Jiayi, et al.
Published: (2024)

AdaptCache: KV Cache Native Storage Hierarchy for Low-Delay and High-Quality Language Model Serving
by: Feng, Shaoting, et al.
Published: (2025)

EVICPRESS: Joint KV-Cache Compression and Eviction for Efficient LLM Serving
by: Feng, Shaoting, et al.
Published: (2025)

Towards More Economical Context-Augmented LLM Generation by Reusing Stored KV Cache
by: Li, Hanchen, et al.
Published: (2025)

DroidSpeak: KV Cache Sharing for Cross-LLM Communication and Multi-LLM Serving
by: Liu, Yuhan, et al.
Published: (2024)

VeriCache: Turning Lossy KV Cache into Lossless LLM Inference
by: Yao, Jiayi, et al.
Published: (2026)

Eloquent: A More Robust Transmission Scheme for LLM Token Streaming
by: Li, Hanchen, et al.
Published: (2024)

LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference
by: Liu, Yuhan, et al.
Published: (2025)

OneAdapt: Fast Configuration Adaptation for Video Analytics Applications via Backpropagation
by: Du, Kuntai, et al.
Published: (2023)

ShadowServe: Interference-Free KV Cache Fetching for Distributed Prefix Caching
by: Xiang, Xingyu, et al.
Published: (2025)

METIS: Fast Quality-Aware RAG Systems with Configuration Adaptation
by: Ray, Siddhant, et al.
Published: (2024)

FastCache: Optimizing Multimodal LLM Serving through Lightweight KV-Cache Compression Framework
by: Zhu, Jianian, et al.
Published: (2025)

StreamKV: Streaming Video Question-Answering with Segment-based KV Cache Retrieval and Compression
by: Chen, Yilong, et al.
Published: (2025)

HyperRAG: Enhancing Quality-Efficiency Tradeoffs in Retrieval-Augmented Generation with Reranker KV-Cache Reuse
by: An, Yuwei, et al.
Published: (2025)

Do Large Language Models Need a Content Delivery Network?
by: Cheng, Yihua, et al.
Published: (2024)

Adaptive KV Cache Reuse for Fast Long-Context LLM Serving
by: li, Fei, et al.
Published: (2026)

The Pitfalls of KV Cache Compression
by: Chen, Alex, et al.
Published: (2025)

Decouple and Cache: KV Cache Construction for Streaming Video Understanding
by: Pang, Zhanzhong, et al.
Published: (2026)

Training Transformers for KV Cache Compressibility
by: Gelberg, Yoav, et al.
Published: (2026)

Lossless KV Cache Compression to 2%
by: Yang, Zhen, et al.
Published: (2024)

KVSculpt: KV Cache Compression as Distillation
by: Jiang, Bo, et al.
Published: (2026)

MuKV: Multi-Grained KV Cache Compression for Long Streaming Video Question-Answering
by: Xiao, Junbin, et al.
Published: (2026)

FastKV: Decoupling of Context Reduction and KV Cache Compression for Prefill-Decoding Acceleration
by: Jo, Dongwon, et al.
Published: (2025)

ReasonCache: Accelerating Large Reasoning Model Serving through KV Cache Sharing
by: Chen, Kaiwen, et al.
Published: (2025)

CacheFlow: Efficient LLM Serving with 3D-Parallel KV Cache Restoration
by: Nian, Sean, et al.
Published: (2026)

KVServe: Service-Aware KV Cache Compression for Communication-Efficient Disaggregated LLM Serving
by: Liu, Zedong, et al.
Published: (2026)

Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Live
by: Li, Hanchen, et al.
Published: (2025)

Understanding the Physics of Key-Value Cache Compression for LLMs through Attention Dynamics
by: Ananthanarayanan, Samhruth, et al.
Published: (2026)

Oaken: Fast and Efficient LLM Serving with Online-Offline Hybrid KV Cache Quantization
by: Kim, Minsu, et al.
Published: (2025)

KV-RM: Regularizing KV-Cache Movement for Static-Graph LLM Serving
by: Zhong, Zhiqing, et al.
Published: (2026)

EvolKV: Evolutionary KV Cache Compression for LLM Inference
by: Yu, Bohan, et al.
Published: (2025)

MiniCache: KV Cache Compression in Depth Dimension for Large Language Models
by: Liu, Akide, et al.
Published: (2024)

CaliDrop: KV Cache Compression with Calibration
by: Su, Yi, et al.
Published: (2025)

Joint Encoding of KV-Cache Blocks for Scalable LLM Serving
by: Kampeas, Joseph, et al.
Published: (2026)

KVReviver: Reversible KV Cache Compression with Sketch-Based Token Reconstruction
by: Yuan, Aomufei, et al.
Published: (2025)

InfiniPot-V: Memory-Constrained KV Cache Compression for Streaming Video Understanding
by: Kim, Minsoo, et al.
Published: (2025)

Cache Your Prompt When It's Green: Carbon-Aware Caching for Large Language Model Serving
by: Tian, Yuyang, et al.
Published: (2025)

Pyramid Cache: Layer-Adaptive KV Cache Compression with Signature-Based Cold Storage
by: Sergio dj
Published: (2026)

SemantiCache: Efficient KV Cache Compression via Semantic Chunking and Clustered Merging
by: Wu, Shunlong, et al.
Published: (2026)

CommonKV: Compressing KV Cache with Cross-layer Parameter Sharing
by: Wang, Yixuan, et al.
Published: (2025)