Cover Image

Self-Pruned Key-Value Attention: Learning When to Write by Predicting Future Utility

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Szilvasy, Gergely, Faysse, Manuel, Lomeli, Maria, Douze, Matthijs, Mazaré, Pierre-Emmanuel, Cabannes, Loïc, Yih, Wen-tau, Jégou, Hervé
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Computation and Language I.2.6; I.2.7
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Short window attention enables long-term memorization
by: Cabannes, Loïc, et al.
Published: (2025)

Vector search with small radiuses
by: Szilvasy, Gergely, et al.
Published: (2024)

Inference-time sparse attention with asymmetric indexing
by: Mazaré, Pierre-Emmanuel, et al.
Published: (2025)

Stochastic activations
by: Lomeli, Maria, et al.
Published: (2025)

When Does Global Attention Help? A Unified Empirical Study on Atomistic Graph Learning
by: Chowdhury, Arindam, et al.
Published: (2025)

SCULPT: Constraint-Guided Pruned MCTS that Carves Efficient Paths for Mathematical Reasoning
by: Fang, Qitong, et al.
Published: (2026)

Co-Writing with AI, on Human Terms: Aligning Research with User Demands Across the Writing Process
by: Reza, Mohi, et al.
Published: (2025)

When in Doubt, Think Slow: Iterative Reasoning with Latent Imagination
by: Benfeghoul, Martin, et al.
Published: (2024)

The Faiss library
by: Douze, Matthijs, et al.
Published: (2024)

Curb Your Attention: Causal Attention Gating for Robust Trajectory Prediction in Autonomous Driving
by: Ahmadi, Ehsan, et al.
Published: (2024)

On Convex Optimal Value Functions For POSGs
by: Cunha, Rafael F., et al.
Published: (2023)

The Arrival of AGI? When Expert Personas Exceed Expert Benchmarks
by: Mullens, Drake, et al.
Published: (2026)

FastForward Pruning: Efficient LLM Pruning via Single-Step Reinforcement Learning
by: Yuan, Xin, et al.
Published: (2025)

Entropy-Based Measurement of Value Drift and Alignment Work in Large Language Models
by: Fadli, Samih
Published: (2025)

ACE: Exploring Activation Cosine Similarity and Variance for Accurate and Calibration-Efficient LLM Pruning
by: Mi, Zhendong, et al.
Published: (2025)

Predicting Future Actions of Reinforcement Learning Agents
by: Chung, Stephen, et al.
Published: (2024)

Key-Value Means: Transformers with Expandable Block-Recurrent Compressed Memory
by: Goldstein, Daniel, et al.
Published: (2026)

Attention-based sequential recommendation system using multimodal data
by: Oh, Hyungtaik, et al.
Published: (2024)

APP: Accelerated Path Patching with Task-Specific Pruning
by: Andersen, Frauke, et al.
Published: (2025)

Alternating Gradient Flow Utility: A Unified Metric for Structural Pruning and Dynamic Routing in Deep Networks
by: Qian, Tianhao, et al.
Published: (2026)

Ultra-Reduced-Impact-Encased-Logging (URIEL): propose a new method for selective sustainable logging and post-harvest silvicultural treatment in tropical forest using airborne robotics systems
by: Albiero, Daniel, et al.
Published: (2026)

LLM Performance Predictors: Learning When to Escalate in Hybrid Human-AI Moderation Systems
by: Bachar, Or, et al.
Published: (2026)

An Explainable Collaborative Dialogue System using a Theory of Mind
by: Cohen, Philip R., et al.
Published: (2023)

When Reasoning Fails: Evaluating 'Thinking' LLMs for Stock Prediction
by: Sodha, Rakeshkumar H
Published: (2025)

EvoPrune: Early-Stage Visual Token Pruning for Efficient MLLMs
by: Chen, Yuhao, et al.
Published: (2026)

QoSGMAA: A Robust Multi-Order Graph Attention and Adversarial Framework for Sparse QoS Prediction
by: Du, Guanchen, et al.
Published: (2025)

AI and Machine Learning Approaches for Predicting Nanoparticles Toxicity The Critical Role of Physiochemical Properties
by: Yousaf, Iqra
Published: (2024)

Aspect-Based Sentiment Analysis for Future Tourism Experiences: A BERT-MoE Framework for Persian User Reviews
by: Taskooh, Hamidreza Kazemi, et al.
Published: (2026)

CLIP-Joint-Detect: End-to-End Joint Training of Object Detectors with Contrastive Vision-Language Supervision
by: Raoufi, Behnam, et al.
Published: (2025)

Discovering Transformer Circuits via a Hybrid Attribution and Pruning Framework
by: Gu, Hao, et al.
Published: (2025)

EvoPruneDeepTL: An Evolutionary Pruning Model for Transfer Learning based Deep Neural Networks
by: Poyatos, Javier, et al.
Published: (2022)

Robust Confidence Intervals in Stereo Matching using Possibility Theory
by: Malinowski, Roman, et al.
Published: (2024)

Descriptive Collision in Sparse Autoencoder Auto-Interpretability: When One Explanation Describes Many Features
by: McCann, Jordan F.
Published: (2026)

Analysing Factorizations of Action-Value Networks for Cooperative Multi-Agent Reinforcement Learning
by: Castellini, Jacopo, et al.
Published: (2019)

Who Gets the Kidney? Human-AI Alignment, Indecision, and Moral Values
by: Dickerson, John P., et al.
Published: (2025)

Deployment-Time Reliability of Learned Robot Policies
by: Agia, Christopher
Published: (2026)

Gyan: An Explainable Neuro-Symbolic Language Model
by: Srinivasan, Venkat, et al.
Published: (2026)

When Gradients Collide: Failure Modes of Multi-Objective Prompt Optimization for LLM Judges
by: Darshan, Parth, et al.
Published: (2026)

When Does Content-Based Routing Work? Representation Requirements for Selective Attention in Hybrid Sequence Models
by: Basu, Abhinaba
Published: (2026)

The Hidden Attention of Mamba Models
by: Ali, Ameen, et al.
Published: (2024)