Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2604.24936 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911627792613376 |
|---|---|
| author | Squires, Chandler Ravikumar, Pradeep |
| author_facet | Squires, Chandler Ravikumar, Pradeep |
| contents | Techniques for concept extraction, such as sparse autoencoders and transcoders, aim to extract high-level symbolic concepts from low-level nonsymbolic representations. When these extracted concepts are used for downstream tasks such as model steering and unlearning, it is essential to understand their guarantees, or lack thereof. In this work, we present a unified theoretical framework for unsupervised concept extraction, in which we frame the task of concept extraction as identifying a generative model. We present a general meta-theorem for identifiability, which reduces the problem of establishing identifiability guarantees to the problem of characterizing the intersection of two sets. As we demonstrate on a range of widely-used approaches, this meta-theorem substantially simplifies the task of proving such guarantees, thus paving the way for the development of new, principled approaches for concept extraction. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_24936 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | A Unifying Framework for Unsupervised Concept Extraction Squires, Chandler Ravikumar, Pradeep Machine Learning 68T01 (Primary), 62A99 (Secondary), 62H25 (Secondary) G.3; I.2.6 Techniques for concept extraction, such as sparse autoencoders and transcoders, aim to extract high-level symbolic concepts from low-level nonsymbolic representations. When these extracted concepts are used for downstream tasks such as model steering and unlearning, it is essential to understand their guarantees, or lack thereof. In this work, we present a unified theoretical framework for unsupervised concept extraction, in which we frame the task of concept extraction as identifying a generative model. We present a general meta-theorem for identifiability, which reduces the problem of establishing identifiability guarantees to the problem of characterizing the intersection of two sets. As we demonstrate on a range of widely-used approaches, this meta-theorem substantially simplifies the task of proving such guarantees, thus paving the way for the development of new, principled approaches for concept extraction. |
| title | A Unifying Framework for Unsupervised Concept Extraction |
| topic | Machine Learning 68T01 (Primary), 62A99 (Secondary), 62H25 (Secondary) G.3; I.2.6 |
| url | https://arxiv.org/abs/2604.24936 |