KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inference

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Xing, Xing, Zeyu, Li, Yiming, Qu, Linping, Zhen, Hui-Ling, Liu, Wulong, Yao, Yiwu, Pan, Sinno Jialin, Yuan, Mingxuan
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Internet

https://arxiv.org/abs/2502.04420