Cover Image

Investigating Sensitive Directions in GPT-2: An Improved Baseline and Comparative Analysis of SAEs

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lee, Daniel J., Heimersheim, Stefan
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

You can remove GPT2's LayerNorm by fine-tuning
by: Heimersheim, Stefan
Published: (2024)

Sanity Checks for Sparse Autoencoders: Do SAEs Beat Random Baselines?
by: Korznikov, Anton, et al.
Published: (2026)

Evaluating Synthetic Activations composed of SAE Latents in GPT-2
by: Giglemiani, Giorgi, et al.
Published: (2024)

How to use and interpret activation patching
by: Heimersheim, Stefan, et al.
Published: (2024)

Tokenized SAEs: Disentangling SAE Reconstructions
by: Dooms, Thomas, et al.
Published: (2025)

Transformers Don't Need LayerNorm at Inference Time: Scaling LayerNorm Removal to GPT-2 XL and the Implications for Mechanistic Interpretability
by: Baroni, Luca, et al.
Published: (2025)

Residual Stream Analysis with Multi-Layer SAEs
by: Lawson, Tim, et al.
Published: (2024)

Distribution-Aware Feature Selection for SAEs
by: Oozeer, Narmeen, et al.
Published: (2025)

The Rate-Distortion-Polysemanticity Tradeoff in SAEs
by: Mencattini, Tommaso, et al.
Published: (2026)

Analyzing (In)Abilities of SAEs via Formal Languages
by: Menon, Abhinav, et al.
Published: (2024)

Interpretability in Parameter Space: Minimizing Mechanistic Description Length with Attribution-based Parameter Decomposition
by: Braun, Dan, et al.
Published: (2025)

Evolution of SAE Features Across Layers in LLMs
by: Balcells, Daniel, et al.
Published: (2024)

Interpretability as Compression: Reconsidering SAE Explanations of Neural Activations with MDL-SAEs
by: Ayonrinde, Kola, et al.
Published: (2024)

Detecting Strategic Deception Using Linear Probes
by: Goldowsky-Dill, Nicholas, et al.
Published: (2025)

The Obfuscation Atlas: Mapping Where Honesty Emerges in RLVR with Deception Probes
by: Taufeeque, Mohammad, et al.
Published: (2026)

Ablating Archetypes: The Stability of Archetypal SAEs is an Artifact of Initialization and Metric Design
by: Brzozowski, Michał, et al.
Published: (2026)

Resa: Transparent Reasoning Models via SAEs
by: Wang, Shangshang, et al.
Published: (2025)

Can SAEs reveal and mitigate racial biases of LLMs in healthcare?
by: Ahsan, Hiba, et al.
Published: (2025)

Features Emerge as Discrete States: The First Application of SAEs to 3D Representations
by: Miao, Albert, et al.
Published: (2025)

SAEs Are Good for Steering -- If You Select the Right Features
by: Arad, Dana, et al.
Published: (2025)

Teach Old SAEs New Domain Tricks with Boosting
by: Koriagin, Nikita, et al.
Published: (2025)

SAEs $\textit{Can}$ Improve Unlearning: Dynamic Sparse Autoencoder Guardrails for Precision Unlearning in LLMs
by: Muhamed, Aashiq, et al.
Published: (2025)

Position: Mechanistic Interpretability Should Prioritize Feature Consistency in SAEs
by: Song, Xiangchen, et al.
Published: (2025)

Benchmarking Deception Probes via Black-to-White Performance Boosts
by: Parrack, Avi, et al.
Published: (2025)

Characterizing stable regions in the residual stream of LLMs
by: Janiak, Jett, et al.
Published: (2024)

SCALAR: Benchmarking SAE Interaction Sparsity in Toy LLMs
by: Fillingham, Sean P., et al.
Published: (2025)

Mechanistic Interpretability with SAEs: Probing Religion, Violence, and Geography in Large Language Models
by: Simbeck, Katharina, et al.
Published: (2025)

Improving Equity in Health Modeling with GPT4-Turbo Generated Synthetic Data: A Comparative Study
by: Smolyak, Daniel, et al.
Published: (2024)

Concept Influence: Leveraging Interpretability to Improve Performance and Efficiency in Training Data Attribution
by: Kowal, Matthew, et al.
Published: (2026)

Compared to What? Baselines and Metrics for Counterfactual Prompting
by: Yang, Zihao, et al.
Published: (2026)

Improved Baselines with Visual Instruction Tuning
by: Liu, Haotian, et al.
Published: (2023)

Using Degeneracy in the Loss Landscape for Mechanistic Interpretability
by: Bushnaq, Lucius, et al.
Published: (2024)

MetaSAEs: Joint Training with a Decomposability Penalty Produces More Atomic Sparse Autoencoder Latents
by: Levinson, Matthew
Published: (2026)

ERM++: An Improved Baseline for Domain Generalization
by: Teterwak, Piotr, et al.
Published: (2023)

A Neural Framework for Generalized Causal Sensitivity Analysis
by: Frauen, Dennis, et al.
Published: (2023)

On the Failure of Topic-Matched Contrast Baselines in Multi-Directional Refusal Abliteration
by: Petrov, Valentin
Published: (2026)

Amortizing Causal Sensitivity Analysis via Prior Data-Fitted Networks
by: Javurek, Emil, et al.
Published: (2026)

Evaluation of GPT-3 for Anti-Cancer Drug Sensitivity Prediction
by: Chowdhury, Shaika, et al.
Published: (2023)

A Comparative Study of NAFNet Baselines for Image Restoration
by: Esaulov, Vladislav, et al.
Published: (2025)

Improved Baselines with Representation Autoencoders
by: Singh, Jaskirat, et al.
Published: (2026)