Cover Image

WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains More

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yue, Yuxuan, Yuan, Zhihang, Duanmu, Haojie, Zhou, Sifan, Wu, Jianlong, Nie, Liqiang
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SKVQ: Sliding-window Key and Value Cache Quantization for Large Language Models
by: Duanmu, Haojie, et al.
Published: (2024)

PCDVQ: Enhancing Vector Quantization for Large Language Models via Polar Coordinate Decoupling
by: Yue, Yuxuan, et al.
Published: (2025)

I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Models
by: Hu, Xing, et al.
Published: (2024)

MambaQuant: Quantizing the Mamba Family with Variance Aligned Rotation Methods
by: Xu, Zukang, et al.
Published: (2025)

WeightedKV: Attention Scores Weighted Key-Value Cache Merging for Large Language Models
by: Yuan, Jian, et al.
Published: (2025)

GSQ-Tuning: Group-Shared Exponents Integer in Fully Quantized Training for LLMs On-Device Fine-tuning
by: Zhou, Sifan, et al.
Published: (2025)

NQKV: A KV Cache Quantization Scheme Based on Normal Distribution Characteristics
by: Cai, Zhihang, et al.
Published: (2025)

MxMoE: Mixed-precision Quantization for MoE with Accuracy and Performance Co-Design
by: Duanmu, Haojie, et al.
Published: (2025)

Parallel Key-Value Cache Fusion for Position Invariant RAG
by: Oh, Philhoon, et al.
Published: (2025)

RWKVQuant: Quantizing the RWKV Family with Proxy Guided Hybrid of Scalar and Vector Quantization
by: Xu, Chen, et al.
Published: (2025)

GWQ: Gradient-Aware Weight Quantization for Large Language Models
by: Shao, Yihua, et al.
Published: (2024)

TableCache: Primary Foreign Key Guided KV Cache Precomputation for Low Latency Text-to-SQL
by: Su, Jinbo, et al.
Published: (2026)

OstQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fitting
by: Hu, Xing, et al.
Published: (2025)

Enhancing Emotional Generation Capability of Large Language Models via Emotional Chain-of-Thought
by: Li, Zaijing, et al.
Published: (2024)

LLaVE: Large Language and Vision Embedding Models with Hardness-Weighted Contrastive Learning
by: Lan, Zhibin, et al.
Published: (2025)

ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Models
by: Yuan, Zhihang, et al.
Published: (2023)

ROSAQ: Rotation-based Saliency-Aware Weight Quantization for Efficiently Compressing Large Language Models
by: Yoon, Junho, et al.
Published: (2025)

MoEQuant: Enhancing Quantization for Mixture-of-Experts Large Language Models via Expert-Balanced Sampling and Affinity Guidance
by: Hu, Xing, et al.
Published: (2025)

ROPO: Robust Preference Optimization for Large Language Models
by: Liang, Xize, et al.
Published: (2024)

LaCache: Ladder-Shaped KV Caching for Efficient Long-Context Modeling of Large Language Models
by: Shi, Dachuan, et al.
Published: (2025)

Evaluating Quantized Large Language Models
by: Li, Shiyao, et al.
Published: (2024)

Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models
by: Liu, Ruikang, et al.
Published: (2025)

Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models
by: Röttger, Paul, et al.
Published: (2024)

Rethinking Key-Value Cache Compression Techniques for Large Language Model Serving
by: Gao, Wei, et al.
Published: (2025)

AMQ: Enabling AutoML for Mixed-precision Weight-Only Quantization of Large Language Models
by: Lee, Sangjun, et al.
Published: (2025)

dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Caching
by: Liu, Zhiyuan, et al.
Published: (2025)

MiniCache: KV Cache Compression in Depth Dimension for Large Language Models
by: Liu, Akide, et al.
Published: (2024)

Quantized Large Language Models in Biomedical Natural Language Processing: Evaluation and Recommendation
by: Zhan, Zaifu, et al.
Published: (2025)

Evaluating and Enhancing Large Language Models for Conversational Reasoning on Knowledge Graphs
by: Huang, Yuxuan
Published: (2023)

6Bit-Diffusion: Inference-Time Mixed-Precision Quantization for Video Diffusion Models
by: Su, Rundong, et al.
Published: (2026)

Confabulation: The Surprising Value of Large Language Model Hallucinations
by: Sui, Peiqi, et al.
Published: (2024)

The Super Weight in Large Language Models
by: Yu, Mengxia, et al.
Published: (2024)

Weights-Rotated Preference Optimization for Large Language Models
by: Yang, Chenxu, et al.
Published: (2025)

Less is More for RAG: Information Gain Pruning for Generator-Aligned Reranking and Evidence Selection
by: Song, Zhipeng, et al.
Published: (2026)

Embedding Domain Knowledge for Large Language Models via Reinforcement Learning from Augmented Generation
by: Nie, Chaojun, et al.
Published: (2025)

ThinK: Thinner Key Cache by Query-Driven Pruning
by: Xu, Yuhui, et al.
Published: (2024)

MQuant: Unleashing the Inference Potential of Multimodal Large Language Models via Full Static Quantization
by: Yu, JiangYong, et al.
Published: (2025)

On the Compressibility of Quantized Large Language Models
by: Mao, Yu, et al.
Published: (2024)

Orca: Enhancing Role-Playing Abilities of Large Language Models by Integrating Personality Traits
by: Huang, Yuxuan
Published: (2024)

When Quantization Affects Confidence of Large Language Models?
by: Proskurina, Irina, et al.
Published: (2024)