Cover Image

Low-Rank Adapting Models for Sparse Autoencoders

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chen, Matthew, Engels, Joshua, Tegmark, Max
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Decomposing The Dark Matter of Sparse Autoencoders
by: Engels, Joshua, et al.
Published: (2024)

Are Sparse Autoencoders Useful? A Case Study in Sparse Probing
by: Kantamneni, Subhash, et al.
Published: (2025)

Efficient Dictionary Learning with Switch Sparse Autoencoders
by: Mudide, Anish, et al.
Published: (2024)

The Geometry of Concepts: Sparse Autoencoder Feature Structure
by: Li, Yuxiao, et al.
Published: (2024)

Not All Language Model Features Are One-Dimensionally Linear
by: Engels, Joshua, et al.
Published: (2024)

Scaling Laws For Scalable Oversight
by: Engels, Joshua, et al.
Published: (2025)

Towards Understanding Distilled Reasoning Models: A Representational Approach
by: Baek, David D., et al.
Published: (2025)

Language Models Represent Space and Time
by: Gurnee, Wes, et al.
Published: (2023)

Language Models Use Trigonometry to Do Addition
by: Kantamneni, Subhash, et al.
Published: (2025)

Dense SAE Latents Are Features, Not Bugs
by: Sun, Xiaoqing, et al.
Published: (2025)

GenEFT: Understanding Statics and Dynamics of Model Generalization via Effective Theory
by: Baek, David D., et al.
Published: (2024)

A Neural Scaling Law from Lottery Ticket Ensembling
by: Liu, Ziming, et al.
Published: (2023)

Harmonic Loss Trains Interpretable AI Models
by: Baek, David D., et al.
Published: (2025)

Do Two AI Scientists Agree?
by: Fu, Xinghong, et al.
Published: (2025)

OptPDE: Discovering Novel Integrable Systems via AI-Human Collaboration
by: Kantamneni, Subhash, et al.
Published: (2024)

On the creation of narrow AI: hierarchy and nonlocality of neural network skills
by: Michaud, Eric J., et al.
Published: (2025)

Finding Belief Geometries with Sparse Autoencoders
by: Levinson, Matthew
Published: (2026)

Resurrecting the Salmon: Rethinking Mechanistic Interpretability with Domain-Specific Sparse Autoencoders
by: O'Neill, Charles, et al.
Published: (2025)

Dynamic Low-Rank Sparse Adaptation for Large Language Models
by: Huang, Weizhong, et al.
Published: (2025)

Investigating Representation Universality: Case Study on Genealogical Representations
by: Baek, David D., et al.
Published: (2024)

Steering Language Model Refusal with Sparse Autoencoders
by: O'Brien, Kyle, et al.
Published: (2024)

A Resource Model For Neural Scaling Law
by: Song, Jinyeop, et al.
Published: (2024)

Rank Reduction Autoencoders
by: Mounayer, Jad, et al.
Published: (2024)

The Quantization Model of Neural Scaling
by: Michaud, Eric J., et al.
Published: (2023)

How Do Transformers "Do" Physics? Investigating the Simple Harmonic Oscillator
by: Kantamneni, Subhash, et al.
Published: (2024)

Ensembling Sparse Autoencoders
by: Gadgil, Soham, et al.
Published: (2025)

Neural Thermodynamic Laws for Large Language Model Training
by: Liu, Ziming, et al.
Published: (2025)

Do Sparse Autoencoders Identify Reasoning Features in Language Models?
by: Ma, George, et al.
Published: (2026)

Interpretable Reward Model via Sparse Autoencoder
by: Zhang, Shuyi, et al.
Published: (2025)

Training Superior Sparse Autoencoders for Instruct Models
by: Li, Jiaming, et al.
Published: (2025)

Variational Rank Reduction Autoencoders
by: Mounayer, Jad, et al.
Published: (2025)

Sparse Autoencoders for Low-$N$ Protein Function Prediction and Design
by: Tsui, Darin, et al.
Published: (2025)

Learning on Transformers is Provable Low-Rank and Sparse: A One-layer Analysis
by: Li, Hongkang, et al.
Published: (2024)

Preference Instability in Reward Models: Detection and Mitigation via Sparse Autoencoders
by: Liu, Shunchang, et al.
Published: (2026)

Analysis of Variational Sparse Autoencoders
by: Baker, Zachary, et al.
Published: (2025)

Toward Identifiable Sparse Autoencoders
by: Nelson, Walter, et al.
Published: (2026)

Probing the Representational Power of Sparse Autoencoders in Vision Models
by: Olson, Matthew Lyle, et al.
Published: (2025)

Analyzing Hierarchical Structure in Vision Models with Sparse Autoencoders
by: Olson, Matthew Lyle, et al.
Published: (2025)

Nonconvex Approach for Sparse and Low-Rank Constrained Models with Dual Momentum
by: Wu, Cho-Ying, et al.
Published: (2019)

SALAAD: Sparse And Low-Rank Adaptation via ADMM for Large Language Model Inference
by: Ma, Hao, et al.
Published: (2026)