Image de couverture de livre

KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inference

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Li, Xing, Xing, Zeyu, Li, Yiming, Qu, Linping, Zhen, Hui-Ling, Liu, Wulong, Yao, Yiwu, Pan, Sinno Jialin, Yuan, Mingxuan
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Beyond Speedup -- Utilizing KV Cache for Sampling and Reasoning
par: Xing, Zeyu, et autres
Publié: (2026)

SVDq: 1.25-bit and 410x Key Cache Compression for LLM Attention
par: Yankun, Hong, et autres
Publié: (2025)

MixPE: Quantization and Hardware Co-design for Efficient LLM Inference
par: Zhang, Yu, et autres
Publié: (2024)

KVmix: Gradient-Based Layer Importance-Aware Mixed-Precision Quantization for KV Cache
par: Li, Fei, et autres
Publié: (2025)

WindowQuant: Mixed-Precision KV Cache Quantization based on Window-Level Similarity for VLMs Inference Optimization
par: Tao, Wei, et autres
Publié: (2026)

AsymKV: Enabling 1-Bit Quantization of KV Cache with Layer-Wise Asymmetric Quantization Configurations
par: Tao, Qian, et autres
Publié: (2024)

AttentionPredictor: Temporal Patterns Matter for KV Cache Compression
par: Yang, Qingyue, et autres
Publié: (2025)

TurboAngle: Near-Lossless KV Cache Compression via Uniform Angle Quantization
par: Patel, Dipkumar
Publié: (2026)

Lossless KV Cache Compression to 2%
par: Yang, Zhen, et autres
Publié: (2024)

GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM
par: Kang, Hao, et autres
Publié: (2024)

FreeKV: Boosting KV Cache Retrieval for Efficient LLM Inference
par: Liu, Guangda, et autres
Publié: (2025)

VeriCache: Turning Lossy KV Cache into Lossless LLM Inference
par: Yao, Jiayi, et autres
Publié: (2026)

KVSharer: Efficient Inference via Layer-Wise Dissimilar KV Cache Sharing
par: Yang, Yifei, et autres
Publié: (2024)

Analytical FFN-to-MoE Restructuring via Activation Pattern Analysis
par: Pei, Zehua, et autres
Publié: (2025)

eOptShrinkQ: Near-Lossless KV Cache Compression Through Optimal Spectral Denoising and Quantization
par: Su, Pei-Chun
Publié: (2026)

KeepKV: Achieving Periodic Lossless KV Cache Compression for Efficient LLM Inference
par: Tian, Yuxuan, et autres
Publié: (2025)

From Pruning to Grafting: Dynamic Knowledge Redistribution via Learnable Layer Fusion
par: Pei, Zehua, et autres
Publié: (2024)

MixKVQ: Query-Aware Mixed-Precision KV Cache Quantization for Long-Context Reasoning
par: Zhang, Tao, et autres
Publié: (2025)

TriAxialKV: Toward Extreme Low-Precision KV-Cache Quantization for Agentic Inference Tasks
par: Shen, Hanzhang, et autres
Publié: (2026)

AlignedKV: Reducing Memory Access of KV-Cache with Precision-Aligned Quantization
par: Tan, Yifan, et autres
Publié: (2024)

RateQuant: Optimal Mixed-Precision KV Cache Quantization via Rate-Distortion Theory
par: Zuo, Fei, et autres
Publié: (2026)

Channel-Wise Mixed-Precision Quantization for Large Language Models
par: Chen, Zihan, et autres
Publié: (2024)

CONF-KV: Confidence-Aware KV Cache Eviction with Mixed-Precision Storage for Long-Horizon LLM
par: Li, Yubo, et autres
Publié: (2026)

WindowKV: Task-Adaptive Group-Wise KV Cache Window Selection for Efficient LLM Inference
par: Zuo, Youhui, et autres
Publié: (2025)

No Token Left Behind: Reliable KV Cache Compression via Importance-Aware Mixed Precision Quantization
par: Yang, June Yong, et autres
Publié: (2024)

ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference
par: Liu, Xiang, et autres
Publié: (2025)

Mix-QViT: Mixed-Precision Vision Transformer Quantization Driven by Layer Importance and Quantization Sensitivity
par: Ranjan, Navin, et autres
Publié: (2025)

Kitty: Accurate and Efficient 2-bit KV Cache Quantization with Dynamic Channel-wise Precision Boost
par: Xia, Haojun, et autres
Publié: (2025)

MiniKV: Pushing the Limits of LLM Inference via 2-Bit Layer-Discriminative KV Cache
par: Sharma, Akshat, et autres
Publié: (2024)

XQuant: Achieving Ultra-Low Bit KV Cache Quantization with Cross-Layer Compression
par: Yang, Haoqi, et autres
Publié: (2025)

FocuSFT: Bilevel Optimization for Dilution-Aware Long-Context Fine-Tuning
par: Pei, Zehua, et autres
Publié: (2026)

Verilog-Evolve: Feedback-Driven and Skill-Evolving Verilog Generation
par: Pei, Zehua, et autres
Publié: (2026)

PM-KVQ: Progressive Mixed-precision KV Cache Quantization for Long-CoT LLMs
par: Liu, Tengxuan, et autres
Publié: (2025)

Accurate KV Cache Quantization with Outlier Tokens Tracing
par: Su, Yi, et autres
Publié: (2025)

DepthKV: Layer-Dependent KV Cache Pruning for Long-Context LLM Inference
par: Dehghanighobadi, Zahra, et autres
Publié: (2026)

PreMoE: Proactive Inference for Efficient Mixture-of-Experts
par: Pei, Zehua, et autres
Publié: (2025)

MorphServe: Efficient and Workload-Aware LLM Serving via Runtime Quantized Layer Swapping and KV Cache Resizing
par: Su, Zhaoyuan, et autres
Publié: (2025)

Value-Driven Mixed-Precision Quantization for Patch-Based Inference on Microcontrollers
par: Tao, Wei, et autres
Publié: (2024)

MomentKV: Closing the Directional Gap in KV Cache Eviction for Long-Context Inference
par: Li, Yu, et autres
Publié: (2026)

LLMCache: Layer-Wise Caching Strategies for Accelerated Reuse in Transformer Inference
par: Bansal, Harsh Vardhan
Publié: (2025)