Cover Image

Decomposing The Dark Matter of Sparse Autoencoders

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Engels, Joshua, Riggs, Logan, Tegmark, Max
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Low-Rank Adapting Models for Sparse Autoencoders
by: Chen, Matthew, et al.
Published: (2025)

Are Sparse Autoencoders Useful? A Case Study in Sparse Probing
by: Kantamneni, Subhash, et al.
Published: (2025)

Efficient Dictionary Learning with Switch Sparse Autoencoders
by: Mudide, Anish, et al.
Published: (2024)

The Geometry of Concepts: Sparse Autoencoder Feature Structure
by: Li, Yuxiao, et al.
Published: (2024)

Scaling Laws For Scalable Oversight
by: Engels, Joshua, et al.
Published: (2025)

Not All Language Model Features Are One-Dimensionally Linear
by: Engels, Joshua, et al.
Published: (2024)

Dense SAE Latents Are Features, Not Bugs
by: Sun, Xiaoqing, et al.
Published: (2025)

Towards Understanding Distilled Reasoning Models: A Representational Approach
by: Baek, David D., et al.
Published: (2025)

Decomposition of Small Transformer Models
by: Christensen, Casper L., et al.
Published: (2025)

Language Models Use Trigonometry to Do Addition
by: Kantamneni, Subhash, et al.
Published: (2025)

Language Models Represent Space and Time
by: Gurnee, Wes, et al.
Published: (2023)

A Neural Scaling Law from Lottery Ticket Ensembling
by: Liu, Ziming, et al.
Published: (2023)

Decoding Dark Matter: Specialized Sparse Autoencoders for Interpreting Rare Concepts in Foundation Models
by: Muhamed, Aashiq, et al.
Published: (2024)

GenEFT: Understanding Statics and Dynamics of Model Generalization via Effective Theory
by: Baek, David D., et al.
Published: (2024)

OptPDE: Discovering Novel Integrable Systems via AI-Human Collaboration
by: Kantamneni, Subhash, et al.
Published: (2024)

Do Two AI Scientists Agree?
by: Fu, Xinghong, et al.
Published: (2025)

On the creation of narrow AI: hierarchy and nonlocality of neural network skills
by: Michaud, Eric J., et al.
Published: (2025)

Resurrecting the Salmon: Rethinking Mechanistic Interpretability with Domain-Specific Sparse Autoencoders
by: O'Neill, Charles, et al.
Published: (2025)

Investigating Representation Universality: Case Study on Genealogical Representations
by: Baek, David D., et al.
Published: (2024)

Harmonic Loss Trains Interpretable AI Models
by: Baek, David D., et al.
Published: (2025)

How Do Transformers "Do" Physics? Investigating the Simple Harmonic Oscillator
by: Kantamneni, Subhash, et al.
Published: (2024)

MetaSAEs: Joint Training with a Decomposability Penalty Produces More Atomic Sparse Autoencoder Latents
by: Levinson, Matthew
Published: (2026)

Ensembling Sparse Autoencoders
by: Gadgil, Soham, et al.
Published: (2025)

Toward Identifiable Sparse Autoencoders
by: Nelson, Walter, et al.
Published: (2026)

Analysis of Variational Sparse Autoencoders
by: Baker, Zachary, et al.
Published: (2025)

SDQ: Sparse Decomposed Quantization for LLM Inference
by: Jeong, Geonhwa, et al.
Published: (2024)

Sparse Autoencoders, Again?
by: Lu, Yin, et al.
Published: (2025)

Knowledge Offloading: Decomposing LLMs into Sparse Backbones and Memory Modules
by: Galliamov, Karim, et al.
Published: (2026)

Training on Documents About Monitoring Leads to CoT Obfuscation
by: Haskins, Reilly, et al.
Published: (2026)

Disentangling Dense Embeddings with Sparse Autoencoders
by: O'Neill, Charles, et al.
Published: (2024)

Transcoders Beat Sparse Autoencoders for Interpretability
by: Paulo, Gonçalo, et al.
Published: (2025)

Evaluating Sparse Autoencoders for Monosemantic Representation
by: Fereidouni, Moghis, et al.
Published: (2025)

A Resource Model For Neural Scaling Law
by: Song, Jinyeop, et al.
Published: (2024)

The Remarkable Robustness of LLMs: Stages of Inference?
by: Lad, Vedang, et al.
Published: (2024)

The Quantization Model of Neural Scaling
by: Michaud, Eric J., et al.
Published: (2023)

Are Sparse Autoencoder Benchmarks Reliable?
by: Chanin, David
Published: (2026)

Neural Thermodynamic Laws for Large Language Model Training
by: Liu, Ziming, et al.
Published: (2025)

Steering Language Model Refusal with Sparse Autoencoders
by: O'Brien, Kyle, et al.
Published: (2024)

Interpreting Attention Layer Outputs with Sparse Autoencoders
by: Kissane, Connor, et al.
Published: (2024)

Attribution-Guided Distillation of Matryoshka Sparse Autoencoders
by: Martin-Linares, Cristina P., et al.
Published: (2025)