Cover Image

Not All Tokens Are Worth Caching: Learning Semantic-Aware Eviction for LLM Prefix Caches

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Fang, Shaoke, Li, Ziang, Wu, Wenfei, Ji, Jiatong, Liu, Qingsong, Pu, Ruizhi
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Distributed, Parallel, and Cluster Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

TokenLake: A Unified Segment-level Prefix Cache Pool for Fine-grained Elastic Long-Context LLM Serving
by: Wu, Bingyang, et al.
Published: (2025)

PrefixWall: Mitigating Prefix Caching Side Channels in Shared LLM Systems
by: Pennas, Panagiotis Georgios, et al.
Published: (2026)

10Cache: Heterogeneous Resource-Aware Tensor Caching and Migration for LLM Training
by: Afroz, Sabiha, et al.
Published: (2025)

ShadowServe: Interference-Free KV Cache Fetching for Distributed Prefix Caching
by: Xiang, Xingyu, et al.
Published: (2025)

RcLLM: Accelerating Generative Recommendation via Beyond-Prefix KV Caching
by: Zhao, Zhan, et al.
Published: (2026)

Marconi: Prefix Caching for the Era of Hybrid LLMs
by: Pan, Rui, et al.
Published: (2024)

CacheFlow: Efficient LLM Serving with 3D-Parallel KV Cache Restoration
by: Nian, Sean, et al.
Published: (2026)

KVFlow: Efficient Prefix Caching for Accelerating LLM-Based Multi-Agent Workflows
by: Pan, Zaifeng, et al.
Published: (2025)

TokenDance: Scaling Multi-Agent LLM Serving via Collective KV Cache Sharing
by: Bian, Zhuohang, et al.
Published: (2026)

Decentralized Federated Learning with Model Caching on Mobile Agents
by: Wang, Xiaoyu, et al.
Published: (2024)

FedCache: A Knowledge Cache-driven Federated Learning Architecture for Personalized Edge Intelligence
by: Wu, Zhiyuan, et al.
Published: (2023)

InstCache: A Predictive Cache for LLM Serving
by: Zou, Longwei, et al.
Published: (2024)

Cortex: Achieving Low-Latency, Cost-Efficient Remote Data Access For LLM via Semantic-Aware Knowledge Caching
by: Ruan, Chaoyi, et al.
Published: (2025)

TDC-Cache: A Trustworthy Decentralized Cooperative Caching Framework for Web3.0
by: Chen, Jinyu, et al.
Published: (2025)

KVPR: Efficient LLM Inference with I/O-Aware KV Cache Partial Recomputation
by: Jiang, Chaoyi, et al.
Published: (2024)

Towards Efficient Key-Value Cache Management for Prefix Prefilling in LLM Inference
by: Zhu, Yue, et al.
Published: (2025)

MorphServe: Efficient and Workload-Aware LLM Serving via Runtime Quantized Layer Swapping and KV Cache Resizing
by: Su, Zhaoyuan, et al.
Published: (2025)

PerCache: Predictive Hierarchical Cache for RAG Applications on Mobile Devices
by: Liu, Kaiwei, et al.
Published: (2025)

FastCache: Optimizing Multimodal LLM Serving through Lightweight KV-Cache Compression Framework
by: Zhu, Jianian, et al.
Published: (2025)

Cost-Efficient LLM Serving in the Cloud: VM Selection with KV Cache Offloading
by: Kim, Kihyun, et al.
Published: (2025)

HACK: Homomorphic Acceleration via Compression of the Key-Value Cache for Disaggregated LLM Inference
by: Zhang, Zeyu, et al.
Published: (2025)

Efficient LLM Inference with Activation Checkpointing and Hybrid Caching
by: Lee, Sanghyeon, et al.
Published: (2025)

TinyServe: Query-Aware Cache Selection for Efficient LLM Serving
by: Liu, Dong, et al.
Published: (2025)

BatchLLM: Optimizing Large Batched LLM Inference with Global Prefix Sharing and Throughput-oriented Token Batching
by: Zheng, Zhen, et al.
Published: (2024)

Efficient Unified Caching for Accelerating Heterogeneous AI Workloads
by: Wang, Tianze, et al.
Published: (2025)

Adaptive K-PackCache: Cost-Centric Data Caching in Cloud
by: Sarkar, Suvarthi, et al.
Published: (2025)

Caching Aided Multi-Tenant Serverless Computing
by: Qiao, Chu, et al.
Published: (2024)

Cache Your Prompt When It's Green: Carbon-Aware Caching for Large Language Model Serving
by: Tian, Yuyang, et al.
Published: (2025)

Efficient Remote KV Cache Reuse with GPU-native Video Codec
by: Mi, Liang, et al.
Published: (2026)

Harnessing Your DRAM and SSD for Sustainable and Accessible LLM Inference with Mixed-Precision and Multi-level Caching
by: Peng, Jie, et al.
Published: (2024)

CaBaFL: Asynchronous Federated Learning via Hierarchical Cache and Feature Balance
by: Xia, Zeke, et al.
Published: (2024)

Kavier: Exploring Performance, Sustainability, and Efficiency of LLM Ecosystems under Inference through Cache-Aware Discrete-Event Simulation
by: Nicolae, Radu, et al.
Published: (2026)

CARM Tool: Cache-Aware Roofline Model Automatic Benchmarking and Application Analysis
by: Morgado, José, et al.
Published: (2026)

CacheFL: Privacy-Preserving and Efficient Federated Cache Model Fine-Tuning for Vision-Language Models
by: Yi, Mengjun, et al.
Published: (2025)

MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool
by: Hu, Cunchen, et al.
Published: (2024)

KVComp: A High-Performance, LLM-Aware, Lossy Compression Framework for KV Cache
by: Jiang, Bo, et al.
Published: (2025)

LLM-dCache: Improving Tool-Augmented LLMs with GPT-Driven Localized Data Caching
by: Singh, Simranjit, et al.
Published: (2024)

Adaptive KV Cache Reuse for Fast Long-Context LLM Serving
by: li, Fei, et al.
Published: (2026)

SHARe-KAN: Post-Training Vector Quantization for Cache-Resident KAN Inference
by: Smith, Jeff
Published: (2025)

Irminsul: MLA-Native Position-Independent Caching for Agentic LLM Serving
by: Ma, Bole, et al.
Published: (2026)