Image de couverture de livre

Accelerating LLM Inference Throughput via Asynchronous KV Cache Prefetching

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Dong, Yanhao, Miao, Yubo, Li, Weinan, Zheng, Xiao, Wang, Chao, Wu, Jiesheng, Lyu, Feng
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

CONF-KV: Confidence-Aware KV Cache Eviction with Mixed-Precision Storage for Long-Horizon LLM
par: Li, Yubo, et autres
Publié: (2026)

RelayCaching: Accelerating LLM Collaboration via Decoding KV Cache Reuse
par: Geng, Yingsheng, et autres
Publié: (2026)

ALISA: Accelerating Large Language Model Inference via Sparsity-Aware KV Caching
par: Zhao, Youpeng, et autres
Publié: (2024)

CacheClip: Accelerating RAG with Effective KV Cache Reuse
par: Yang, Bin, et autres
Publié: (2025)

SentenceKV: Efficient LLM Inference via Sentence-Level Semantic KV Caching
par: Zhu, Yuxuan, et autres
Publié: (2025)

FreeKV: Boosting KV Cache Retrieval for Efficient LLM Inference
par: Liu, Guangda, et autres
Publié: (2025)

Get More with LESS: Synthesizing Recurrence with KV Cache Compression for Efficient LLM Inference
par: Dong, Harry, et autres
Publié: (2024)

SmallKV: Small Model Assisted Compensation of KV Cache Compression for Efficient LLM Inference
par: Zhao, Yi, et autres
Publié: (2025)

Online Scheduling for LLM Inference with KV Cache Constraints
par: Jaillet, Patrick, et autres
Publié: (2025)

TTKV: Temporal-Tiered KV Cache for Long-Context LLM Inference
par: Dzikanyanga, Gradwell, et autres
Publié: (2026)

KeepKV: Achieving Periodic Lossless KV Cache Compression for Efficient LLM Inference
par: Tian, Yuxuan, et autres
Publié: (2025)

KV Cache Transform Coding for Compact Storage in LLM Inference
par: Staniszewski, Konrad, et autres
Publié: (2025)

RAP: KV-Cache Compression via RoPE-Aligned Pruning
par: Xin, Jihao, et autres
Publié: (2026)

ReasonCache: Accelerating Large Reasoning Model Serving through KV Cache Sharing
par: Chen, Kaiwen, et autres
Publié: (2025)

CoKV: Optimizing KV Cache Allocation via Cooperative Game
par: Sun, Qiheng, et autres
Publié: (2025)

TokenSelect: Efficient Long-Context Inference and Length Extrapolation for LLMs via Dynamic Token-Level KV Cache Selection
par: Wu, Wei, et autres
Publié: (2024)

ClusterKV: Manipulating LLM KV Cache in Semantic Space for Recallable Compression
par: Liu, Guangda, et autres
Publié: (2024)

ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference
par: Sun, Hanshi, et autres
Publié: (2024)

MadaKV: Adaptive Modality-Perception KV Cache Eviction for Efficient Multimodal Long-Context Inference
par: Li, Kunxi, et autres
Publié: (2025)

Joint Encoding of KV-Cache Blocks for Scalable LLM Serving
par: Kampeas, Joseph, et autres
Publié: (2026)

GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM
par: Kang, Hao, et autres
Publié: (2024)

LouisKV: Efficient KV Cache Retrieval for Long Input-Output Sequences
par: Wu, Wenbo, et autres
Publié: (2025)

The Pitfalls of KV Cache Compression
par: Chen, Alex, et autres
Publié: (2025)

Improving Model Representation and Reducing KV Cache via Skip Connections with First Value Heads
par: Wu, Zhoutong, et autres
Publié: (2025)

KVSharer: Efficient Inference via Layer-Wise Dissimilar KV Cache Sharing
par: Yang, Yifei, et autres
Publié: (2024)

The Residual Stream Is All You Need: On the Redundancy of the KV Cache in Transformer Inference
par: Qasim, Kaleem Ullah, et autres
Publié: (2026)

The Illusion of Equivalence: Systematic FP16 Divergence in KV-Cached Autoregressive Inference
par: Chodavarapu, Ranjith, et autres
Publié: (2026)

A Queueing-Theoretic Framework for Stability Analysis of LLM Inference with KV Cache Memory Constraints
par: Nie, Chengyi, et autres
Publié: (2026)

ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identification
par: He, Yefei, et autres
Publié: (2024)

Leyline: KV Cache Directives for Agentic Inference
par: Ma, Bole, et autres
Publié: (2026)

Crystal-KV: Efficient KV Cache Management for Chain-of-Thought LLMs via Answer-First Principle
par: Wang, Zihan, et autres
Publié: (2026)

KV Packet: Recomputation-Free Context-Independent KV Caching for LLMs
par: Chen, Chuangtao, et autres
Publié: (2026)

CommonKV: Compressing KV Cache with Cross-layer Parameter Sharing
par: Wang, Yixuan, et autres
Publié: (2025)

EVICPRESS: Joint KV-Cache Compression and Eviction for Efficient LLM Serving
par: Feng, Shaoting, et autres
Publié: (2025)

ReCalKV: Low-Rank KV Cache Compression via Head Reordering and Offline Calibration
par: Yan, Xianglong, et autres
Publié: (2025)

How Much Cache Does Reasoning Need? Depth-Cache Tradeoffs in KV-Compressed Transformers
par: Wang, Xiao
Publié: (2026)

Palu: Compressing KV-Cache with Low-Rank Projection
par: Chang, Chi-Chih, et autres
Publié: (2024)

AgenticCache: Cache-Driven Asynchronous Planning for Embodied AI Agents
par: Kim, Hojoon, et autres
Publié: (2026)

Cache What Lasts: Token Retention for Memory-Bounded KV Cache in LLMs
par: Bui, Ngoc, et autres
Publié: (2025)

HCAttention: Extreme KV Cache Compression via Heterogeneous Attention Computing for LLMs
par: Yang, Dongquan, et autres
Publié: (2025)