Image de couverture de livre

SmallKV: Small Model Assisted Compensation of KV Cache Compression for Efficient LLM Inference

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zhao, Yi, Peng, Yajuan, Nguyen, Cam-Tu, Li, Zuchao, Wang, Xiaoliang, Zhao, Hai, Fu, Xiaoming
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

TrigReason: Trigger-Based Collaboration between Small and Large Reasoning Models
par: Zhao, Yi, et autres
Publié: (2026)

KeepKV: Achieving Periodic Lossless KV Cache Compression for Efficient LLM Inference
par: Tian, Yuxuan, et autres
Publié: (2025)

LAVa: Layer-wise KV Cache Eviction with Dynamic Budget Allocation
par: Shen, Yiqun, et autres
Publié: (2025)

FreeKV: Boosting KV Cache Retrieval for Efficient LLM Inference
par: Liu, Guangda, et autres
Publié: (2025)

HybridKV: Hybrid KV Cache Compression for Efficient Multimodal Large Language Model Inference
par: Zeng, Bowen, et autres
Publié: (2026)

ClusterKV: Manipulating LLM KV Cache in Semantic Space for Recallable Compression
par: Liu, Guangda, et autres
Publié: (2024)

GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM
par: Kang, Hao, et autres
Publié: (2024)

Revisiting Multimodal KV Cache Compression: A Frequency-Domain-Guided Outlier-KV-Aware Approach
par: Yang, Yaoxin, et autres
Publié: (2025)

Get More with LESS: Synthesizing Recurrence with KV Cache Compression for Efficient LLM Inference
par: Dong, Harry, et autres
Publié: (2024)

SparKV: Overhead-Aware KV Cache Loading for Efficient On-Device LLM Inference
par: Liu, Hongyao, et autres
Publié: (2026)

SentenceKV: Efficient LLM Inference via Sentence-Level Semantic KV Caching
par: Zhu, Yuxuan, et autres
Publié: (2025)

Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference
par: Feng, Yuan, et autres
Publié: (2024)

OBCache: Optimal Brain KV Cache Pruning for Efficient Long-Context LLM Inference
par: Gu, Yuzhe, et autres
Publié: (2025)

R-KV: Redundancy-aware KV Cache Compression for Reasoning Models
par: Cai, Zefan, et autres
Publié: (2025)

The Pitfalls of KV Cache Compression
par: Chen, Alex, et autres
Publié: (2025)

Comparative Characterization of KV Cache Management Strategies for LLM Inference
par: Mamo, Oteo, et autres
Publié: (2026)

KVSharer: Efficient Inference via Layer-Wise Dissimilar KV Cache Sharing
par: Yang, Yifei, et autres
Publié: (2024)

DepthKV: Layer-Dependent KV Cache Pruning for Long-Context LLM Inference
par: Dehghanighobadi, Zahra, et autres
Publié: (2026)

BUZZ: Beehive-structured Sparse KV Cache with Segmented Heavy Hitters for Efficient LLM Inference
par: Zhao, Junqi, et autres
Publié: (2024)

Q-Filters: Leveraging QK Geometry for Efficient KV Cache Compression
par: Godey, Nathan, et autres
Publié: (2025)

CommonKV: Compressing KV Cache with Cross-layer Parameter Sharing
par: Wang, Yixuan, et autres
Publié: (2025)

EVICPRESS: Joint KV-Cache Compression and Eviction for Efficient LLM Serving
par: Feng, Shaoting, et autres
Publié: (2025)

Lossless KV Cache Compression to 2%
par: Yang, Zhen, et autres
Publié: (2024)

Moment-KV: Momentum-Based Decode-Time KV Cache Compression for Long Generation
par: Jana, Soumyadeep, et autres
Publié: (2026)

PackKV: Reducing KV Cache Memory Footprint through LLM-Aware Lossy Compression
par: Jiang, Bo, et autres
Publié: (2025)

VL-Cache: Sparsity and Modality-Aware KV Cache Compression for Vision-Language Model Inference Acceleration
par: Tu, Dezhan, et autres
Publié: (2024)

FairKV: Balancing Per-Head KV Cache for Fast Multi-GPU Inference
par: Zhao, Bingzhe, et autres
Publié: (2025)

PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling
par: Cai, Zefan, et autres
Publié: (2024)

Online Scheduling for LLM Inference with KV Cache Constraints
par: Jaillet, Patrick, et autres
Publié: (2025)

HeteroCache: A Dynamic Retrieval Approach to Heterogeneous KV Cache Compression for Long-Context LLM Inference
par: Shi, Zhiyuan, et autres
Publié: (2026)

AhaKV: Adaptive Holistic Attention-Driven KV Cache Eviction for Efficient Inference of Large Language Models
par: Gu, Yifeng, et autres
Publié: (2025)

KVSculpt: KV Cache Compression as Distillation
par: Jiang, Bo, et autres
Publié: (2026)

KVServe: Service-Aware KV Cache Compression for Communication-Efficient Disaggregated LLM Serving
par: Liu, Zedong, et autres
Publié: (2026)

DeltaKV: Residual-Based KV Cache Compression via Long-Range Similarity
par: Hao, Jitai, et autres
Publié: (2026)

OjaKV: Context-Aware Online Low-Rank KV Cache Compression
par: Zhu, Yuxuan, et autres
Publié: (2025)

Shadow in the Cache: Unveiling and Mitigating Privacy Risks of KV-cache in LLM Inference
par: Luo, Zhifan, et autres
Publié: (2025)

SkipKV: Selective Skipping of KV Generation and Storage for Efficient Inference with Large Reasoning Models
par: Tian, Jiayi, et autres
Publié: (2025)

AirCache: Activating Inter-modal Relevancy KV Cache Compression for Efficient Large Vision-Language Model Inference
par: Huang, Kai, et autres
Publié: (2025)

KV Cache Transform Coding for Compact Storage in LLM Inference
par: Staniszewski, Konrad, et autres
Publié: (2025)

KVReviver: Reversible KV Cache Compression with Sketch-Based Token Reconstruction
par: Yuan, Aomufei, et autres
Publié: (2025)