Image de couverture de livre

Q-Filters: Leveraging QK Geometry for Efficient KV Cache Compression

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Godey, Nathan, Devoto, Alessio, Zhao, Yu, Scardapane, Simone, Minervini, Pasquale, de la Clergerie, Éric, Sagot, Benoît
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

A Simple and Effective $L_2$ Norm-Based Strategy for KV Cache Compression
par: Devoto, Alessio, et autres
Publié: (2024)

Anisotropy Is Inherent to Self-Attention in Transformers
par: Godey, Nathan, et autres
Publié: (2024)

Why do small language models underperform? Studying Language Model Saturation via the Softmax Bottleneck
par: Godey, Nathan, et autres
Publié: (2024)

On the Scaling Laws of Geographical Representation in Language Models
par: Godey, Nathan, et autres
Publié: (2024)

Biomed-Enriched: A Biomedical Dataset Enriched with LLMs for Pretraining and Extracting Rare and Hidden Content
par: Touchent, Rian, et autres
Publié: (2025)

Gaperon: A Peppered English-French Generative Language Model Suite
par: Godey, Nathan, et autres
Publié: (2025)

Adaptive Computation Modules: Granular Conditional Computation For Efficient Inference
par: Wójcik, Bartosz, et autres
Publié: (2023)

Expected Attention: KV Cache Compression by Estimating Attention from Future Queries Distribution
par: Devoto, Alessio, et autres
Publié: (2025)

Adaptive Layer Selection for Efficient Vision Transformer Fine-Tuning
par: Devoto, Alessio, et autres
Publié: (2024)

Conditional computation in neural networks: principles and research trends
par: Scardapane, Simone, et autres
Publié: (2024)

Patent Representation Learning via Self-supervision
par: Zuo, You, et autres
Publié: (2025)

PatentEval: Understanding Errors in Patent Generation
par: Zuo, You, et autres
Publié: (2024)

CamemBERT 2.0: A Smarter French Language Model Aged to Perfection
par: Antoun, Wissam, et autres
Publié: (2024)

Attention Sinks in Diffusion Language Models
par: Rulli, Maximo Eduardo, et autres
Publié: (2025)

Q-KVComm: Efficient Multi-Agent Communication Via Adaptive KV Cache Compression
par: Kriuk, Boris, et autres
Publié: (2025)

Class incremental learning with probability dampening and cascaded gated classifier
par: Pomponi, Jary, et autres
Publié: (2024)

EchoKV: Efficient KV Cache Compression via Similarity-Based Reconstruction
par: Ji, Shiyu, et autres
Publié: (2026)

CamemBERT-bio: Leveraging Continual Pre-training for Cost-Effective Models on French Biomedical Data
par: Touchent, Rian, et autres
Publié: (2023)

Lost in Backpropagation: The LM Head is a Gradient Bottleneck
par: Godey, Nathan, et autres
Publié: (2026)

EvolKV: Evolutionary KV Cache Compression for LLM Inference
par: Yu, Bohan, et autres
Publié: (2025)

SemantiCache: Efficient KV Cache Compression via Semantic Chunking and Clustered Merging
par: Wu, Shunlong, et autres
Publié: (2026)

Compactor: Calibrated Query-Agnostic KV Cache Compression with Approximate Leverage Scores
par: Chari, Vivek, et autres
Publié: (2025)

ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference
par: Liu, Xiang, et autres
Publié: (2025)

KV-Compress: Paged KV-Cache Compression with Variable Compression Rates per Attention Head
par: Rehg, Isaac
Publié: (2024)

Universal Properties of Activation Sparsity in Modern Large Language Models
par: Szatkowski, Filip, et autres
Publié: (2025)

LongFlow: Efficient KV Cache Compression for Reasoning Models
par: Su, Yi, et autres
Publié: (2026)

Lossless KV Cache Compression to 2%
par: Yang, Zhen, et autres
Publié: (2024)

A Causal Language Modeling Detour Improves Encoder Continued Pretraining
par: Touchent, Rian, et autres
Publié: (2026)

CaliDrop: KV Cache Compression with Calibration
par: Su, Yi, et autres
Publié: (2025)

KeepKV: Achieving Periodic Lossless KV Cache Compression for Efficient LLM Inference
par: Tian, Yuxuan, et autres
Publié: (2025)

CodeComp: Structural KV Cache Compression for Agentic Coding
par: Chen, Qiujiang, et autres
Publié: (2026)

RocketKV: Accelerating Long-Context LLM Inference via Two-Stage KV Cache Compression
par: Behnam, Payman, et autres
Publié: (2025)

KVSculpt: KV Cache Compression as Distillation
par: Jiang, Bo, et autres
Publié: (2026)

RazorAttention: Efficient KV Cache Compression Through Retrieval Heads
par: Tang, Hanlin, et autres
Publié: (2024)

DeltaKV: Residual-Based KV Cache Compression via Long-Range Similarity
par: Hao, Jitai, et autres
Publié: (2026)

NestedKV: Nested Memory Routing for Long-Context KV Cache Compression
par: Chen, Hong, et autres
Publié: (2026)

R-KV: Redundancy-aware KV Cache Compression for Reasoning Models
par: Cai, Zefan, et autres
Publié: (2025)

OjaKV: Context-Aware Online Low-Rank KV Cache Compression
par: Zhu, Yuxuan, et autres
Publié: (2025)

Analysing the Residual Stream of Language Models Under Knowledge Conflicts
par: Zhao, Yu, et autres
Publié: (2024)

Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineering
par: Zhao, Yu, et autres
Publié: (2024)