Cover Image

Intrinsically Interpretable Attention via Sparse Post-Training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Draye, Florent, Lei, Anson, Pan, Hsiao-Ru, Posner, Ingmar, Schölkopf, Bernhard
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Identifying Intervenable and Interpretable Features via Orthogonality Regularization
by: Miller, Moritz, et al.
Published: (2026)

SPARTAN: A Sparse Transformer World Model Attending to What Matters
by: Lei, Anson, et al.
Published: (2024)

Compete and Compose: Learning Independent Mechanisms for Modular World Models
by: Lei, Anson, et al.
Published: (2024)

Skill or Luck? Return Decomposition via Advantage Functions
by: Pan, Hsiao-Ru, et al.
Published: (2024)

Disentangling Dynamical Systems: Causal Representation Learning Meets Local Sparse Attention
by: Baumgartner, Markus W., et al.
Published: (2026)

World Models via Policy-Guided Trajectory Diffusion
by: Rigter, Marc, et al.
Published: (2023)

Learning to Reason Efficiently with A* Post-Training
by: Opedal, Andreas, et al.
Published: (2026)

Post-Training Sparse Attention with Double Sparsity
by: Yang, Shuo, et al.
Published: (2024)

XQC: Well-conditioned Optimization Accelerates Deep Reinforcement Learning
by: Palenicek, Daniel, et al.
Published: (2025)

Causality can systematically address the monsters under the bench(marks)
by: Leeb, Felix, et al.
Published: (2025)

A Probabilistic Model Behind Self-Supervised Learning
by: Bizeul, Alice, et al.
Published: (2024)

Out-of-Variable Generalization for Discriminative Models
by: Guo, Siyuan, et al.
Published: (2023)

Computational Arbitrage in AI Model Markets
by: Olmedo, Ricardo, et al.
Published: (2026)

Reparameterized LLM Training via Orthogonal Equivalence Transformation
by: Qiu, Zeju, et al.
Published: (2025)

Learning Interpretable Concepts: Unifying Causal Representation Learning and Foundation Models
by: Rajendran, Goutham, et al.
Published: (2024)

Grasp-MPC: Closed-Loop Visual Grasping via Value-Guided Model Predictive Control
by: Yamada, Jun, et al.
Published: (2025)

DITTO: Offline Imitation Learning with World Models
by: DeMoss, Branton, et al.
Published: (2023)

Conformal Generative Modeling with Improved Sample Efficiency through Sequential Greedy Filtering
by: Kladny, Klaus-Rudolf, et al.
Published: (2024)

SparseBalance: Load-Balanced Long Context Training with Dynamic Sparse Attention
by: Xu, Hongtao, et al.
Published: (2026)

Riemannian Networks over Full-Rank Correlation Matrices
by: Chen, Ziheng, et al.
Published: (2026)

Preference Elicitation for Offline Reinforcement Learning
by: Pace, Alizée, et al.
Published: (2024)

Provable Privacy with Non-Private Pre-Processing
by: Hu, Yaxi, et al.
Published: (2024)

Skill Learning via Policy Diversity Yields Identifiable Representations for Reinforcement Learning
by: Reizinger, Patrik, et al.
Published: (2025)

Lil: Less is Less When Applying Post-Training Sparse-Attention Algorithms in Long-Decode Stage
by: Hu, Junhao, et al.
Published: (2026)

A Mathematical Theory of Top-$k$ Sparse Attention via Total Variation Distance
by: Tzachristas, Georgios, et al.
Published: (2025)

Riemannian Batch Normalization: A Gyro Approach
by: Chen, Ziheng, et al.
Published: (2025)

Offline Adaptation of Quadruped Locomotion using Diffusion Models
by: O'Mahoney, Reece, et al.
Published: (2024)

vAttention: Verified Sparse Attention
by: Desai, Aditya, et al.
Published: (2025)

Learning Nonlinear Causal Reductions to Explain Reinforcement Learning Policies
by: Kekić, Armin, et al.
Published: (2025)

Natural Building Blocks for Structured World Models: Theory, Evidence, and Scaling
by: Da Costa, Lancelot, et al.
Published: (2025)

Identifiable Exchangeable Mechanisms for Causal Structure and Representation Learning
by: Reizinger, Patrik, et al.
Published: (2024)

Deep Backtracking Counterfactuals for Causally Compliant Explanations
by: Kladny, Klaus-Rudolf, et al.
Published: (2023)

SeerAttention-R: Sparse Attention Adaptation for Long Reasoning
by: Gao, Yizhao, et al.
Published: (2025)

Train One Sparse Autoencoder Across Multiple Sparsity Budgets to Preserve Interpretability and Accuracy
by: Balagansky, Nikita, et al.
Published: (2025)

Intrinsic Reward Policy Optimization for Sparse-Reward Environments
by: Cho, Minjae, et al.
Published: (2026)

Towards Interpretable Adversarial Examples via Sparse Adversarial Attack
by: Lin, Fudong, et al.
Published: (2025)

Post-Training Statistical Calibration for Higher Activation Sparsity
by: Chua, Vui Seng, et al.
Published: (2024)

Stargazer: A Scalable Model-Fitting Benchmark Environment for AI Agents under Astrophysical Constraints
by: Liu, Xinge, et al.
Published: (2026)

Improving Large Language Model Safety with Contrastive Representation Learning
by: Simko, Samuel, et al.
Published: (2025)

Counterfactual reasoning: an analysis of in-context emergence
by: Miller, Moritz, et al.
Published: (2025)