Imagen de Portada

PolyKV: A Shared Asymmetrically-Compressed KV Cache Pool for Multi-Agent LLM Inference

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Patel, Ishan, Joshi, Ishan
Formato:	Preprint
Publicado:	2026
Materias:	Machine Learning Computation and Language Distributed, Parallel, and Cluster Computing
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

FDC: Fast KV Dimensionality Compression for Efficient LLM Inference
por: Zhang, Zeyu, et al.
Publicado: (2024)

ForkKV: Scaling Multi-LoRA Agent Serving via Copy-on-Write Disaggregated KV Cache
por: Wang, Shao, et al.
Publicado: (2026)

PrefillShare: A Shared Prefill Module for KV Reuse in Multi-LLM Disaggregated Serving
por: Woo, Sunghyeon, et al.
Publicado: (2026)

Leyline: KV Cache Directives for Agentic Inference
por: Ma, Bole, et al.
Publicado: (2026)

KVPR: Efficient LLM Inference with I/O-Aware KV Cache Partial Recomputation
por: Jiang, Chaoyi, et al.
Publicado: (2024)

KV Cache Compression for Inference Efficiency in LLMs: A Review
por: Liu, Yanyu, et al.
Publicado: (2025)

Cost-Efficient LLM Serving in the Cloud: VM Selection with KV Cache Offloading
por: Kim, Kihyun, et al.
Publicado: (2025)

TokenDance: Scaling Multi-Agent LLM Serving via Collective KV Cache Sharing
por: Bian, Zhuohang, et al.
Publicado: (2026)

SplitZip: Ultra Fast Lossless KV Compression for Disaggregated LLM Serving
por: Guo, Yipin, et al.
Publicado: (2026)

InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management
por: Lee, Wonbeom, et al.
Publicado: (2024)

Efficient Remote KV Cache Reuse with GPU-native Video Codec
por: Mi, Liang, et al.
Publicado: (2026)

ShadowServe: Interference-Free KV Cache Fetching for Distributed Prefix Caching
por: Xiang, Xingyu, et al.
Publicado: (2025)

FlowKV: A Disaggregated Inference Framework with Low-Latency KV Cache Transfer and Load-Aware Scheduling
por: Li, Weiqing, et al.
Publicado: (2025)

MoSKA: Mixture of Shared KV Attention for Efficient Long-Sequence LLM Inference
por: Rhee, Myunghyun, et al.
Publicado: (2025)

KV-Runahead: Scalable Causal LLM Inference by Parallel Key-Value Cache Generation
por: Cho, Minsik, et al.
Publicado: (2024)

MorphServe: Efficient and Workload-Aware LLM Serving via Runtime Quantized Layer Swapping and KV Cache Resizing
por: Su, Zhaoyuan, et al.
Publicado: (2025)

DiffKV: Differentiated Memory Management for Large Language Models with Parallel KV Compaction
por: Zhang, Yanqi, et al.
Publicado: (2024)

PackKV: Reducing KV Cache Memory Footprint through LLM-Aware Lossy Compression
por: Jiang, Bo, et al.
Publicado: (2025)

TraCT: Disaggregated LLM Serving with CXL Shared Memory KV Cache at Rack-Scale
por: Yoon, Dongha, et al.
Publicado: (2025)

FairKV: Balancing Per-Head KV Cache for Fast Multi-GPU Inference
por: Zhao, Bingzhe, et al.
Publicado: (2025)

MatKV: Trading Compute for Flash Storage in LLM Inference
por: Shin, Kun-Woo, et al.
Publicado: (2025)

FastCache: Optimizing Multimodal LLM Serving through Lightweight KV-Cache Compression Framework
por: Zhu, Jianian, et al.
Publicado: (2025)

Efficient Multi-Adapter LLM Serving via Cross-Model KV-Cache Reuse with Activated LoRA
por: Li, Allison, et al.
Publicado: (2025)

CacheFlow: Efficient LLM Serving with 3D-Parallel KV Cache Restoration
por: Nian, Sean, et al.
Publicado: (2026)

HACK: Homomorphic Acceleration via Compression of the Key-Value Cache for Disaggregated LLM Inference
por: Zhang, Zeyu, et al.
Publicado: (2025)

Pie: Pooling CPU Memory for LLM Inference
por: Xu, Yi, et al.
Publicado: (2024)

SpecKV: Adaptive Speculative Decoding with Compression-Aware Gamma Selection
por: Shukla, Shikhar
Publicado: (2026)

ContiguousKV: Accelerating LLM Prefill with Granularity-Aligned KV Cache Management
por: Zou, Jing, et al.
Publicado: (2026)

FineServe: Precision-Aware KV Slab and Two-Level Scheduling for Heterogeneous Precision LLM Serving
por: Bin, Kyungmin, et al.
Publicado: (2025)

RcLLM: Accelerating Generative Recommendation via Beyond-Prefix KV Caching
por: Zhao, Zhan, et al.
Publicado: (2026)

KVComp: A High-Performance, LLM-Aware, Lossy Compression Framework for KV Cache
por: Jiang, Bo, et al.
Publicado: (2025)

VL-Cache: Sparsity and Modality-Aware KV Cache Compression for Vision-Language Model Inference Acceleration
por: Tu, Dezhan, et al.
Publicado: (2024)

TokenLake: A Unified Segment-level Prefix Cache Pool for Fine-grained Elastic Long-Context LLM Serving
por: Wu, Bingyang, et al.
Publicado: (2025)

Harnessing Your DRAM and SSD for Sustainable and Accessible LLM Inference with Mixed-Precision and Multi-level Caching
por: Peng, Jie, et al.
Publicado: (2024)

LLM-Pilot: Characterize and Optimize Performance of your LLM Inference Services
por: Łazuka, Małgorzata, et al.
Publicado: (2024)

Adaptive Multi-Objective Tiered Storage Configuration for KV Cache in LLM Service
por: Zheng, Xianzhe, et al.
Publicado: (2026)

FlexLLM: Token-Level Co-Serving of LLM Inference and Finetuning with SLO Guarantees
por: Oliaro, Gabriele, et al.
Publicado: (2024)

The Workload-Router-Pool Architecture for LLM Inference Optimization: A Vision Paper from the vLLM Semantic Router Project
por: Chen, Huamin, et al.
Publicado: (2026)

OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization
por: Zhou, Zhongzhu, et al.
Publicado: (2026)

PipeInfer: Accelerating LLM Inference using Asynchronous Pipelined Speculation
por: Butler, Branden, et al.
Publicado: (2024)