Imagen de Portada

KVShare: An LLM Service System with Efficient and Effective Multi-Tenant KV Cache Reuse

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Yang, Huan, Zhang, Renji, Huang, Mingzhe, Wang, Weijun, Tang, Yin, Li, Yuanchun, Liu, Yunxin, Zhang, Deyu
Formato:	Preprint
Publicado:	2025
Materias:	Computation and Language Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

A First Look At Efficient And Secure On-Device LLM Inference Against KV Leakage
por: Yang, Huan, et al.
Publicado: (2024)

When KV Cache Reuse Fails in Multi-Agent Systems: Cross-Candidate Interaction is Crucial for LLM Judges
por: Liang, Sichu, et al.
Publicado: (2026)

Efficient Remote KV Cache Reuse with GPU-native Video Codec
por: Mi, Liang, et al.
Publicado: (2026)

EFIM: Efficient Serving of LLMs for Infilling Tasks with Improved KV Cache Reuse
por: Guo, Tianyu, et al.
Publicado: (2025)

KVLink: Accelerating Large Language Models via Efficient KV Cache Reuse
por: Yang, Jingbo, et al.
Publicado: (2025)

ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference
por: Liu, Xiang, et al.
Publicado: (2025)

WindowKV: Task-Adaptive Group-Wise KV Cache Window Selection for Efficient LLM Inference
por: Zuo, Youhui, et al.
Publicado: (2025)

FreeKV: Boosting KV Cache Retrieval for Efficient LLM Inference
por: Liu, Guangda, et al.
Publicado: (2025)

Efficient Long-Context LLM Inference via KV Cache Clustering
por: Hu, Jie, et al.
Publicado: (2025)

KeepKV: Achieving Periodic Lossless KV Cache Compression for Efficient LLM Inference
por: Tian, Yuxuan, et al.
Publicado: (2025)

KVDrive: A Holistic Multi-Tier KV Cache Management System for Long-Context LLM Inference
por: Lin, Jian, et al.
Publicado: (2026)

DASH-KV: Accelerating Long-Context LLM Inference via Asymmetric KV Cache Hashing
por: Guo, Jinyu, et al.
Publicado: (2026)

SentenceKV: Efficient LLM Inference via Sentence-Level Semantic KV Caching
por: Zhu, Yuxuan, et al.
Publicado: (2025)

OxyGen: Unified KV Cache Management for VLA Inference under Multi-Task Parallelism
por: Li, Xiangyu, et al.
Publicado: (2026)

CacheClip: Accelerating RAG with Effective KV Cache Reuse
por: Yang, Bin, et al.
Publicado: (2025)

GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM
por: Kang, Hao, et al.
Publicado: (2024)

EchoKV: Efficient KV Cache Compression via Similarity-Based Reconstruction
por: Ji, Shiyu, et al.
Publicado: (2026)

Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference
por: Feng, Yuan, et al.
Publicado: (2024)

GUI-KV: Efficient GUI Agents via KV Cache with Spatio-Temporal Awareness
por: Huang, Kung-Hsiang, et al.
Publicado: (2025)

PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inference
por: Yang, Dongjie, et al.
Publicado: (2024)

SpindleKV: A Novel KV Cache Reduction Method Balancing Both Shallow and Deep Layers
por: Tang, Zicong, et al.
Publicado: (2025)

DroidSpeak: KV Cache Sharing for Cross-LLM Communication and Multi-LLM Serving
por: Liu, Yuhan, et al.
Publicado: (2024)

ZSMerge: Zero-Shot KV Cache Compression for Memory-Efficient Long-Context LLMs
por: Liu, Xin, et al.
Publicado: (2025)

HyperRAG: Enhancing Quality-Efficiency Tradeoffs in Retrieval-Augmented Generation with Reranker KV-Cache Reuse
por: An, Yuwei, et al.
Publicado: (2025)

RazorAttention: Efficient KV Cache Compression Through Retrieval Heads
por: Tang, Hanlin, et al.
Publicado: (2024)

Which Heads Matter for Reasoning? RL-Guided KV Cache Compression
por: Du, Wenjie, et al.
Publicado: (2025)

MEDA: Dynamic KV Cache Allocation for Efficient Multimodal Long-Context Inference
por: Wan, Zhongwei, et al.
Publicado: (2025)

EvolKV: Evolutionary KV Cache Compression for LLM Inference
por: Yu, Bohan, et al.
Publicado: (2025)

MiniKV: Pushing the Limits of LLM Inference via 2-Bit Layer-Discriminative KV Cache
por: Sharma, Akshat, et al.
Publicado: (2024)

VecInfer: Efficient LLM Inference with Low-Bit KV Cache via Outlier-Suppressed Vector Quantization
por: Yao, Dingyu, et al.
Publicado: (2025)

NACL: A General and Effective KV Cache Eviction Framework for LLMs at Inference Time
por: Chen, Yilong, et al.
Publicado: (2024)

QAQ: Quality Adaptive Quantization for LLM KV Cache
por: Dong, Shichen, et al.
Publicado: (2024)

Taming the Fragility of KV Cache Eviction in LLM Inference
por: Feng, Yuan, et al.
Publicado: (2025)

Effectively Compress KV Heads for LLM
por: Yu, Hao, et al.
Publicado: (2024)

ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference
por: Sun, Hanshi, et al.
Publicado: (2024)

LongFlow: Efficient KV Cache Compression for Reasoning Models
por: Su, Yi, et al.
Publicado: (2026)

DeltaKV: Residual-Based KV Cache Compression via Long-Range Similarity
por: Hao, Jitai, et al.
Publicado: (2026)

BUZZ: Beehive-structured Sparse KV Cache with Segmented Heavy Hitters for Efficient LLM Inference
por: Zhao, Junqi, et al.
Publicado: (2024)

RelayCaching: Accelerating LLM Collaboration via Decoding KV Cache Reuse
por: Geng, Yingsheng, et al.
Publicado: (2026)

Spotlight Attention: Towards Efficient LLM Generation via Non-linear Hashing-based KV Cache Retrieval
por: Li, Wenhao, et al.
Publicado: (2025)