Copertina

WeightedKV: Attention Scores Weighted Key-Value Cache Merging for Large Language Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Yuan, Jian, He, Ziwei, Bai, Haoli, Leng, Jingwen, Jiang, Bo
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

TreeKV: Smooth Key-Value Cache Compression with Tree Structures
di: He, Ziwei, et al.
Pubblicazione: (2025)

FreqKV: Key-Value Compression in Frequency Domain for Context Window Extension
di: Kai, Jushi, et al.
Pubblicazione: (2025)

Fovea Transformer: Efficient Long-Context Modeling with Structured Fine-to-Coarse Attention
di: He, Ziwei, et al.
Pubblicazione: (2023)

WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains More
di: Yue, Yuxuan, et al.
Pubblicazione: (2024)

AhaKV: Adaptive Holistic Attention-Driven KV Cache Eviction for Efficient Inference of Large Language Models
di: Gu, Yifeng, et al.
Pubblicazione: (2025)

One Size Does Not Fit All: Token-Wise Adaptive Compression for KV Cache
di: Lu, Liming, et al.
Pubblicazione: (2026)

AWM: Accurate Weight-Matrix Fingerprint for Large Language Models
di: Zeng, Boyi, et al.
Pubblicazione: (2025)

IntactKV: Improving Large Language Model Quantization by Keeping Pivot Tokens Intact
di: Liu, Ruikang, et al.
Pubblicazione: (2024)

Key, Value, Compress: A Systematic Exploration of KV Cache Compression Techniques
di: Javidnia, Neusha, et al.
Pubblicazione: (2025)

SKVQ: Sliding-window Key and Value Cache Quantization for Large Language Models
di: Duanmu, Haojie, et al.
Pubblicazione: (2024)

Attention Score is not All You Need for Token Importance Indicator in KV Cache Reduction: Value Also Matters
di: Guo, Zhiyu, et al.
Pubblicazione: (2024)

Beyond Homogeneous Attention: Memory-Efficient LLMs via Fourier-Approximated KV Cache
di: Liu, Xiaoran, et al.
Pubblicazione: (2025)

MiniCache: KV Cache Compression in Depth Dimension for Large Language Models
di: Liu, Akide, et al.
Pubblicazione: (2024)

dKV-Cache: The Cache for Diffusion Language Models
di: Ma, Xinyin, et al.
Pubblicazione: (2025)

Model Tells You Where to Merge: Adaptive KV Cache Merging for LLMs on Long-Context Tasks
di: Wang, Zheng, et al.
Pubblicazione: (2024)

SemantiCache: Efficient KV Cache Compression via Semantic Chunking and Clustered Merging
di: Wu, Shunlong, et al.
Pubblicazione: (2026)

Task-KV: Task-aware KV Cache Optimization via Semantic Differentiation of Attention Heads
di: He, Xingyang, et al.
Pubblicazione: (2025)

KVSculpt: KV Cache Compression as Distillation
di: Jiang, Bo, et al.
Pubblicazione: (2026)

AKVQ-VL: Attention-Aware KV Cache Adaptive 2-Bit Quantization for Vision-Language Models
di: Su, Zunhai, et al.
Pubblicazione: (2025)

KVSink: Understanding and Enhancing the Preservation of Attention Sinks in KV Cache Quantization for LLMs
di: Su, Zunhai, et al.
Pubblicazione: (2025)

Layer-Condensed KV Cache for Efficient Inference of Large Language Models
di: Wu, Haoyi, et al.
Pubblicazione: (2024)

Dataless Knowledge Fusion by Merging Weights of Language Models
di: Jin, Xisen, et al.
Pubblicazione: (2022)

$A^3$: Attention-Aware Accurate KV Cache Fusion for Fast Large Language Model Serving
di: Zhou, Yuechi, et al.
Pubblicazione: (2025)

LaCache: Ladder-Shaped KV Caching for Efficient Long-Context Modeling of Large Language Models
di: Shi, Dachuan, et al.
Pubblicazione: (2025)

KV-Runahead: Scalable Causal LLM Inference by Parallel Key-Value Cache Generation
di: Cho, Minsik, et al.
Pubblicazione: (2024)

KVLink: Accelerating Large Language Models via Efficient KV Cache Reuse
di: Yang, Jingbo, et al.
Pubblicazione: (2025)

A Method for Building Large Language Models with Predefined KV Cache Capacity
di: Yi, Zhonghua, et al.
Pubblicazione: (2024)

G-KV: Decoding-Time KV Cache Eviction with Global Attention
di: Liao, Mengqi, et al.
Pubblicazione: (2025)

FlowKV: Enhancing Multi-Turn Conversational Coherence in LLMs via Isolated Key-Value Cache Management
di: Liu, Xiang, et al.
Pubblicazione: (2025)

Enhancing Essay Scoring with Adversarial Weights Perturbation and Metric-specific AttentionPooling
di: Huang, Jiaxin, et al.
Pubblicazione: (2024)

AttentionPredictor: Temporal Patterns Matter for KV Cache Compression
di: Yang, Qingyue, et al.
Pubblicazione: (2025)

Extend Model Merging from Fine-Tuned to Pre-Trained Large Language Models via Weight Disentanglement
di: Yu, Le, et al.
Pubblicazione: (2024)

Homogeneous Keys, Heterogeneous Values: Exploiting Local KV Cache Asymmetry for Long-Context LLMs
di: Cui, Wanyun, et al.
Pubblicazione: (2025)

EMS: Adaptive Evict-then-Merge Strategy for Head-wise KV Cache Compression Based on Global-Local Importance
di: Li, Yingxin, et al.
Pubblicazione: (2024)

Understanding the Physics of Key-Value Cache Compression for LLMs through Attention Dynamics
di: Ananthanarayanan, Samhruth, et al.
Pubblicazione: (2026)

FlowMM: Cross-Modal Information Flow Guided KV Cache Merging for Efficient Multimodal Context Inference
di: Li, Kunxi, et al.
Pubblicazione: (2025)

KV-Compress: Paged KV-Cache Compression with Variable Compression Rates per Attention Head
di: Rehg, Isaac
Pubblicazione: (2024)

EntropyCache: Decoded Token Entropy Guided KV Caching for Diffusion Language Models
di: Cheong, Minsoo, et al.
Pubblicazione: (2026)

HeteroCache: A Dynamic Retrieval Approach to Heterogeneous KV Cache Compression for Long-Context LLM Inference
di: Shi, Zhiyuan, et al.
Pubblicazione: (2026)

Sparse Attention across Multiple-context KV Cache
di: Cao, Ziyi, et al.
Pubblicazione: (2025)