Image de couverture de livre

KeyDiff: Key Similarity-Based KV Cache Eviction for Long-Context LLM Inference in Resource-Constrained Environments

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Park, Junyoung, Jones, Dalton, Morse, Matthew J, Goel, Raghavv, Lee, Mingu, Lott, Chris
Format:	Preprint
Publié:	2025
Sujets:	Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

CAOTE: KV Cache Selection for LLMs via Attention Output Error-Based Token Eviction
par: Goel, Raghavv, et autres
Publié: (2025)

QUOKA: Query-Oriented KV Selection For Efficient LLM Prefill
par: Jones, Dalton, et autres
Publié: (2026)

Recursive Speculative Decoding: Accelerating LLM Inference via Sampling Without Replacement
par: Jeon, Wonseok, et autres
Publié: (2024)

Efficient Training-Free Multi-Token Prediction via Embedding-Space Probing
par: Goel, Raghavv, et autres
Publié: (2026)

On Speculative Decoding for Multimodal Large Language Models
par: Gagrani, Mukul, et autres
Publié: (2024)

Direct Alignment of Draft Model for Speculative Decoding with Chat-Fine-Tuned LLMs
par: Goel, Raghavv, et autres
Publié: (2024)

Reformulating KV Cache Eviction Problem for Long-Context LLM Inference
par: Mai, Tho, et autres
Publié: (2026)

MomentKV: Closing the Directional Gap in KV Cache Eviction for Long-Context Inference
par: Li, Yu, et autres
Publié: (2026)

IndexMem: Learned KV-Cache Eviction with Latent Memory for Long-Context LLM Inference
par: Yang, Xintong, et autres
Publié: (2026)

Taming the Fragility of KV Cache Eviction in LLM Inference
par: Feng, Yuan, et autres
Publié: (2025)

Spiffy: Multiplying Diffusion LLM Acceleration via Lossless Speculative Decoding
par: Agrawal, Sudhanshu, et autres
Publié: (2025)

A Comparative analysis of Layer-wise Representational Capacity in AR and Diffusion LLMs
par: Goel, Raghavv, et autres
Publié: (2026)

MadaKV: Adaptive Modality-Perception KV Cache Eviction for Efficient Multimodal Long-Context Inference
par: Li, Kunxi, et autres
Publié: (2025)

Compressing KV Cache for Long-Context LLM Inference with Inter-Layer Attention Similarity
par: Ma, Da, et autres
Publié: (2024)

Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference
par: Feng, Yuan, et autres
Publié: (2024)

MPCache: MPC-Friendly KV Cache Eviction for Efficient Private LLM Inference
par: Zeng, Wenxuan, et autres
Publié: (2025)

LLMs Know What to Drop: Self-Attention Guided KV Cache Eviction for Efficient Long-Context Inference
par: Wang, Guangtao, et autres
Publié: (2025)

CONF-KV: Confidence-Aware KV Cache Eviction with Mixed-Precision Storage for Long-Horizon LLM
par: Li, Yubo, et autres
Publié: (2026)

EpiCache: Episodic KV Cache Management for Long-Term Conversation on Resource-Constrained Environments
par: Kim, Minsoo, et autres
Publié: (2025)

Make Each Token Count: Towards Improving Long-Context Performance with KV Cache Eviction
par: Bui, Ngoc, et autres
Publié: (2026)

ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference
par: Sun, Hanshi, et autres
Publié: (2024)

DepthKV: Layer-Dependent KV Cache Pruning for Long-Context LLM Inference
par: Dehghanighobadi, Zahra, et autres
Publié: (2026)

VOCABTRIM: Vocabulary Pruning for Efficient Speculative Decoding in LLMs
par: Goel, Raghavv, et autres
Publié: (2025)

TTKV: Temporal-Tiered KV Cache for Long-Context LLM Inference
par: Dzikanyanga, Gradwell, et autres
Publié: (2026)

Efficient Long-Context LLM Inference via KV Cache Clustering
par: Hu, Jie, et autres
Publié: (2025)

KV-Runahead: Scalable Causal LLM Inference by Parallel Key-Value Cache Generation
par: Cho, Minsik, et autres
Publié: (2024)

EVICPRESS: Joint KV-Cache Compression and Eviction for Efficient LLM Serving
par: Feng, Shaoting, et autres
Publié: (2025)

On the Efficacy of Eviction Policy for Key-Value Constrained Generative Language Model Inference
par: Ren, Siyu, et autres
Publié: (2024)

DASH-KV: Accelerating Long-Context LLM Inference via Asymmetric KV Cache Hashing
par: Guo, Jinyu, et autres
Publié: (2026)

ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference
par: Liu, Xiang, et autres
Publié: (2025)

ForesightKV: Optimizing KV Cache Eviction for Reasoning Models by Learning Long-Term Contribution
par: Dong, Zican, et autres
Publié: (2026)

Fast Forward: Accelerating LLM Prefill with Predictive FFN Sparsity
par: Gautam, Aayush, et autres
Publié: (2026)

HillInfer: Efficient Long-Context LLM Inference on the Edge with Hierarchical KV Eviction using SmartSSD
par: Sun, He, et autres
Publié: (2026)

G-KV: Decoding-Time KV Cache Eviction with Global Attention
par: Liao, Mengqi, et autres
Publié: (2025)

Homogeneous Keys, Heterogeneous Values: Exploiting Local KV Cache Asymmetry for Long-Context LLMs
par: Cui, Wanyun, et autres
Publié: (2025)

XKV: Personalized KV Cache Memory Reduction for Long-Context LLM Inference
par: Li, Weizhuo, et autres
Publié: (2024)

RocketKV: Accelerating Long-Context LLM Inference via Two-Stage KV Cache Compression
par: Behnam, Payman, et autres
Publié: (2025)

KV-Fold: One-Step KV-Cache Recurrence for Long-Context Inference
par: Nadali, Alireza, et autres
Publié: (2026)

NACL: A General and Effective KV Cache Eviction Framework for LLMs at Inference Time
par: Chen, Yilong, et autres
Publié: (2024)

AhaKV: Adaptive Holistic Attention-Driven KV Cache Eviction for Efficient Inference of Large Language Models
par: Gu, Yifeng, et autres
Publié: (2025)