Imagen de Portada

IC-Cache: Efficient Large Language Model Serving via In-context Caching

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Yu, Yifan, Gan, Yu, Sarda, Nikhil, Tsai, Lillian, Shen, Jiaming, Zhou, Yanqi, Krishnamurthy, Arvind, Lai, Fan, Levy, Henry M., Culler, David
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

CacheFlow: Efficient LLM Serving with 3D-Parallel KV Cache Restoration
por: Nian, Sean, et al.
Publicado: (2026)

CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving
por: Liu, Yuhan, et al.
Publicado: (2023)

TinyServe: Query-Aware Cache Selection for Efficient LLM Serving
por: Liu, Dong, et al.
Publicado: (2025)

Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttention
por: Gao, Bin, et al.
Publicado: (2024)

ReasonCache: Accelerating Large Reasoning Model Serving through KV Cache Sharing
por: Chen, Kaiwen, et al.
Publicado: (2025)

EPIC: Efficient Position-Independent Caching for Serving Large Language Models
por: Hu, Junhao, et al.
Publicado: (2024)

CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion
por: Yao, Jiayi, et al.
Publicado: (2024)

ShadowServe: Interference-Free KV Cache Fetching for Distributed Prefix Caching
por: Xiang, Xingyu, et al.
Publicado: (2025)

Cache Your Prompt When It's Green: Carbon-Aware Caching for Large Language Model Serving
por: Tian, Yuyang, et al.
Publicado: (2025)

Prompt Cache: Modular Attention Reuse for Low-Latency Inference
por: Gim, In, et al.
Publicado: (2023)

InstCache: A Predictive Cache for LLM Serving
por: Zou, Longwei, et al.
Publicado: (2024)

MTServe: Efficient Serving for Generative Recommendation Models with Hierarchical Caches
por: Wang, Xin, et al.
Publicado: (2026)

Rethinking Key-Value Cache Compression Techniques for Large Language Model Serving
por: Gao, Wei, et al.
Publicado: (2025)

MEPIC: Memory Efficient Position Independent Caching for LLM Serving
por: Wang, Qian, et al.
Publicado: (2025)

Cache-to-Cache: Direct Semantic Communication Between Large Language Models
por: Fu, Tianyu, et al.
Publicado: (2025)

AdaptCache: KV Cache Native Storage Hierarchy for Low-Delay and High-Quality Language Model Serving
por: Feng, Shaoting, et al.
Publicado: (2025)

LaCache: Ladder-Shaped KV Caching for Efficient Long-Context Modeling of Large Language Models
por: Shi, Dachuan, et al.
Publicado: (2025)

Cachemir: Fully Homomorphic Encrypted Inference of Generative Large Language Model with KV Cache
por: Yu, Ye, et al.
Publicado: (2026)

dKV-Cache: The Cache for Diffusion Language Models
por: Ma, Xinyin, et al.
Publicado: (2025)

EVICPRESS: Joint KV-Cache Compression and Eviction for Efficient LLM Serving
por: Feng, Shaoting, et al.
Publicado: (2025)

One Pool, Two Caches: Adaptive HBM Partitioning for Accelerating Generative Recommender Serving
por: Yu, Wenjun, et al.
Publicado: (2026)

Mell: Memory-Efficient Large Language Model Serving via Multi-GPU KV Cache Management
por: Qianli, Liu, et al.
Publicado: (2025)

LLaMCAT: Optimizing Large Language Model Inference with Cache Arbitration and Throttling
por: Zhou, Zhongchun, et al.
Publicado: (2025)

ToolCaching: Towards Efficient Caching for LLM Tool-calling
por: Zhai, Yi, et al.
Publicado: (2026)

SoundWeaver: Semantic Warm-Starting for Text-to-Audio Diffusion Serving
por: Barik, Ayush, et al.
Publicado: (2026)

Apt-Serve: Adaptive Request Scheduling on Hybrid Cache for Scalable LLM Inference Serving
por: Gao, Shihong, et al.
Publicado: (2025)

SCOPE: Optimizing Key-Value Cache Compression in Long-context Generation
por: Wu, Jialong, et al.
Publicado: (2024)

KVCache Cache in the Wild: Characterizing and Optimizing KVCache Cache at a Large Cloud Provider
por: Wang, Jiahao, et al.
Publicado: (2025)

FastCache: Fast Caching for Diffusion Transformer Through Learnable Linear Approximation
por: Liu, Dong, et al.
Publicado: (2025)

$A^3$: Attention-Aware Accurate KV Cache Fusion for Fast Large Language Model Serving
por: Zhou, Yuechi, et al.
Publicado: (2025)

MorphServe: Efficient and Workload-Aware LLM Serving via Runtime Quantized Layer Swapping and KV Cache Resizing
por: Su, Zhaoyuan, et al.
Publicado: (2025)

DualMap: Enabling Both Cache Affinity and Load Balancing for Distributed LLM Serving
por: Yuan, Ying, et al.
Publicado: (2026)

CacheFL: Privacy-Preserving and Efficient Federated Cache Model Fine-Tuning for Vision-Language Models
por: Yi, Mengjun, et al.
Publicado: (2025)

PolyServe: Efficient Multi-SLO Serving at Scale
por: Zhu, Kan, et al.
Publicado: (2025)

JITServe: SLO-aware LLM Serving with Imprecise Request Information
por: Zhang, Wei, et al.
Publicado: (2025)

Anchor Attention, Small Cache: Code Generation with Large Language Models
por: Zhang, Xiangyu, et al.
Publicado: (2024)

Cache-Craft: Managing Chunk-Caches for Efficient Retrieval-Augmented Generation
por: Agarwal, Shubham, et al.
Publicado: (2025)

FastCache: Optimizing Multimodal LLM Serving through Lightweight KV-Cache Compression Framework
por: Zhu, Jianian, et al.
Publicado: (2025)

Strata: Hierarchical Context Caching for Long Context Language Model Serving
por: Xie, Zhiqiang, et al.
Publicado: (2025)

LayerKV: Optimizing Large Language Model Serving with Layer-wise KV Cache Management
por: Xiong, Yi, et al.
Publicado: (2024)