Cover Image

Identifying Intervenable and Interpretable Features via Orthogonality Regularization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Miller, Moritz, Draye, Florent, Schölkopf, Bernhard
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Intrinsically Interpretable Attention via Sparse Post-Training
by: Draye, Florent, et al.
Published: (2025)

Counterfactual reasoning: an analysis of in-context emergence
by: Miller, Moritz, et al.
Published: (2025)

Reparameterized LLM Training via Orthogonal Equivalence Transformation
by: Qiu, Zeju, et al.
Published: (2025)

Concept Layers: Enhancing Interpretability and Intervenability via LLM Conceptualization
by: Bidusa, Or Raphael, et al.
Published: (2025)

Orthogonal Finetuning Made Scalable
by: Qiu, Zeju, et al.
Published: (2025)

Improving Large Language Model Safety with Contrastive Representation Learning
by: Simko, Samuel, et al.
Published: (2025)

Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization
by: Liu, Weiyang, et al.
Published: (2023)

MathGAP: Out-of-Distribution Evaluation on Problems with Arbitrarily Complex Proofs
by: Opedal, Andreas, et al.
Published: (2024)

Limits of Transformer Language Models on Learning to Compose Algorithms
by: Thomm, Jonathan, et al.
Published: (2024)

Learning to Reason Efficiently with A* Post-Training
by: Opedal, Andreas, et al.
Published: (2026)

Generalized Interpolating Discrete Diffusion
by: von Rütte, Dimitri, et al.
Published: (2025)

Learning Beyond Pattern Matching? Assaying Mathematical Understanding in LLMs
by: Guo, Siyuan, et al.
Published: (2024)

Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable?
by: Méloux, Maxime, et al.
Published: (2025)

ProxySPEX: Inference-Efficient Interpretability via Sparse Feature Interactions in LLMs
by: Butler, Landon, et al.
Published: (2025)

Can Large Language Models Infer Causation from Correlation?
by: Jin, Zhijing, et al.
Published: (2023)

Analyzing the Role of Semantic Representations in the Era of Large Language Models
by: Jin, Zhijing, et al.
Published: (2024)

Do Language Models Exhibit the Same Cognitive Biases in Problem Solving as Human Learners?
by: Opedal, Andreas, et al.
Published: (2024)

Hard Prompts Made Interpretable: Sparse Entropy Regularization for Prompt Tuning with RL
by: Choi, Yunseon, et al.
Published: (2024)

Training-Free Tokenizer Transplantation via Orthogonal Matching Pursuit
by: Goddard, Charles, et al.
Published: (2025)

From Isolation to Entanglement: When Do Interpretability Methods Identify and Disentangle Known Concepts?
by: Mueller, Aaron, et al.
Published: (2025)

Superscopes: Amplifying Internal Feature Representations for Language Model Interpretation
by: Jacobi, Jonathan, et al.
Published: (2025)

Position: Mechanistic Interpretability Should Prioritize Feature Consistency in SAEs
by: Song, Xiangchen, et al.
Published: (2025)

Comparing Feature Importance and Rule Extraction for Interpretability on Text Data
by: Lopardo, Gianluigi, et al.
Published: (2022)

Parameter Efficient Quasi-Orthogonal Fine-Tuning via Givens Rotation
by: Ma, Xinyu, et al.
Published: (2024)

MatryoshkaKV: Adaptive KV Compression via Trainable Orthogonal Projection
by: Lin, Bokai, et al.
Published: (2024)

Computational Arbitrage in AI Model Markets
by: Olmedo, Ricardo, et al.
Published: (2026)

Interpretable Steering of Large Language Models with Feature Guided Activation Additions
by: Soo, Samuel, et al.
Published: (2025)

Quriosity: Analyzing Human Questioning Behavior and Causal Inquiry through Curiosity-Driven Queries
by: Ceraolo, Roberto, et al.
Published: (2024)

On the Emergence and Test-Time Use of Structural Information in Large Language Models
by: Chen, Michelle Chao, et al.
Published: (2026)

Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models
by: Marks, Samuel, et al.
Published: (2024)

Disentangling Task Conflicts in Multi-Task LoRA via Orthogonal Gradient Projection
by: Yang, Ziyu, et al.
Published: (2026)

OrthoRank: Token Selection via Sink Token Orthogonality for Efficient LLM inference
by: Shin, Seungjun, et al.
Published: (2025)

The Geometries of Truth Are Orthogonal Across Tasks
by: Azizian, Waiss, et al.
Published: (2025)

Orthogonal Finetuning for Direct Preference Optimization
by: Yang, Chenxu, et al.
Published: (2024)

Cultural Alien Sampler: Open-ended art generation balancing originality and coherence
by: Artiles, Alejandro H., et al.
Published: (2025)

When Answers Stray from Questions: Hallucination Detection via Question-Answer Orthogonal Decomposition
by: Yao, Siyang, et al.
Published: (2026)

Identifying Features Associated with Bias Against 93 Stigmatized Groups in Language Models and Guardrail Model Safety Mitigation
by: Gueorguieva, Anna-Maria, et al.
Published: (2025)

CLadder: Assessing Causal Reasoning in Language Models
by: Jin, Zhijing, et al.
Published: (2023)

CLT-Forge: A Scalable Library for Cross-Layer Transcoders and Attribution Graphs
by: Draye, Florent, et al.
Published: (2026)

Selection of LLM Fine-Tuning Data based on Orthogonal Rules
by: Li, Xiaomin, et al.
Published: (2024)