Copertina

SKVQ: Sliding-window Key and Value Cache Quantization for Large Language Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Duanmu, Haojie, Yuan, Zhihang, Li, Xiuhong, Duan, Jiangfei, Zhang, Xingcheng, Lin, Dahua
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

MxMoE: Mixed-precision Quantization for MoE with Accuracy and Performance Co-Design
di: Duanmu, Haojie, et al.
Pubblicazione: (2025)

WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains More
di: Yue, Yuxuan, et al.
Pubblicazione: (2024)

MuxServe: Flexible Spatial-Temporal Multiplexing for Multiple LLM Serving
di: Duan, Jiangfei, et al.
Pubblicazione: (2024)

NQKV: A KV Cache Quantization Scheme Based on Normal Distribution Characteristics
di: Cai, Zhihang, et al.
Pubblicazione: (2025)

PolarQuant: Leveraging Polar Transformation for Efficient Key Cache Quantization and Decoding Acceleration
di: Wu, Songhao, et al.
Pubblicazione: (2025)

Learning to Evict from Key-Value Cache
di: Moschella, Luca, et al.
Pubblicazione: (2026)

The Policy Cliff: A Theoretical Analysis of Reward-Policy Maps in Large Language Models
di: Xu, Xingcheng
Pubblicazione: (2025)

Cache Me If You Must: Adaptive Key-Value Quantization for Large Language Models
di: Shutova, Alina, et al.
Pubblicazione: (2025)

I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Models
di: Hu, Xing, et al.
Pubblicazione: (2024)

InnerQ: Hardware-Aware Tuning-Free Quantization of KV Cache for Large Language Models
di: Hosseini, Sayed Mohammadreza Tayaranian, et al.
Pubblicazione: (2026)

FBQuant: FeedBack Quantization for Large Language Models
di: Liu, Yijiang, et al.
Pubblicazione: (2025)

SampleAttention: Near-Lossless Acceleration of Long Context LLM Inference with Adaptive Structured Sparse Attention
di: Zhu, Qianchao, et al.
Pubblicazione: (2024)

LaCache: Ladder-Shaped KV Caching for Efficient Long-Context Modeling of Large Language Models
di: Shi, Dachuan, et al.
Pubblicazione: (2025)

Reducing Transformer Key-Value Cache Size with Cross-Layer Attention
di: Brandon, William, et al.
Pubblicazione: (2024)

BAQ: Efficient Bit Allocation Quantization for Large Language Models
di: Zhang, Chao, et al.
Pubblicazione: (2025)

dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Caching
di: Liu, Zhiyuan, et al.
Pubblicazione: (2025)

CBQ: Cross-Block Quantization for Large Language Models
di: Ding, Xin, et al.
Pubblicazione: (2023)

OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models
di: Shao, Wenqi, et al.
Pubblicazione: (2023)

BoostStep: Boosting mathematical capability of Large Language Models via improved single-step reasoning
di: Zhang, Beichen, et al.
Pubblicazione: (2025)

FlashDecoding++: Faster Large Language Model Inference on GPUs
di: Hong, Ke, et al.
Pubblicazione: (2023)

NanoCP: Request-Level Dynamic Context Parallelism for Data-Expert Parallel Decoding
di: Chen, Jiefei, et al.
Pubblicazione: (2026)

Cache & Distil: Optimising API Calls to Large Language Models
di: Ramírez, Guillem, et al.
Pubblicazione: (2023)

Quantization of Large Language Models with an Overdetermined Basis
di: Merkulov, Daniil, et al.
Pubblicazione: (2024)

LCQ: Low-Rank Codebook based Quantization for Large Language Models
di: Cai, Wen-Pu, et al.
Pubblicazione: (2024)

CSKV: Training-Efficient Channel Shrinking for KV Cache in Long-Context Scenarios
di: Wang, Luning, et al.
Pubblicazione: (2024)

MiniCache: KV Cache Compression in Depth Dimension for Large Language Models
di: Liu, Akide, et al.
Pubblicazione: (2024)

Rethinking Key-Value Cache Compression Techniques for Large Language Model Serving
di: Gao, Wei, et al.
Pubblicazione: (2025)

How Quantization Shapes Bias in Large Language Models
di: Marcuzzi, Federico, et al.
Pubblicazione: (2025)

Sliding Window Attention Training for Efficient Large Language Models
di: Fu, Zichuan, et al.
Pubblicazione: (2025)

Off-Policy Value-Based Reinforcement Learning for Large Language Models
di: Wang, Peng-Yuan, et al.
Pubblicazione: (2026)

On the Compressibility of Quantized Large Language Models
di: Mao, Yu, et al.
Pubblicazione: (2024)

KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache
di: Liu, Zirui, et al.
Pubblicazione: (2024)

Scaling Laws for Post Training Quantized Large Language Models
di: Xu, Zifei, et al.
Pubblicazione: (2024)

Extreme Compression of Large Language Models via Additive Quantization
di: Egiazarian, Vage, et al.
Pubblicazione: (2024)

SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models
di: Huang, Wei, et al.
Pubblicazione: (2024)

Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttention
di: Gao, Bin, et al.
Pubblicazione: (2024)

RSAVQ: Riemannian Sensitivity-Aware Vector Quantization for Large Language Models
di: Xu, Zukang, et al.
Pubblicazione: (2025)

ApiQ: Finetuning of 2-Bit Quantized Large Language Model
di: Liao, Baohao, et al.
Pubblicazione: (2024)

QuIP: 2-Bit Quantization of Large Language Models With Guarantees
di: Chee, Jerry, et al.
Pubblicazione: (2023)

Optimizing Large Language Model Training Using FP4 Quantization
di: Wang, Ruizhe, et al.
Pubblicazione: (2025)