Semantic Regexes: Auto-Interpreting LLM Features with a Structured Language
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910034530664448 |
|---|---|
| author | Boggust, Angie Ren, Donghao Assogba, Yannick Moritz, Dominik Satyanarayan, Arvind Hohman, Fred |
| author_facet | Boggust, Angie Ren, Donghao Assogba, Yannick Moritz, Dominik Satyanarayan, Arvind Hohman, Fred |
| contents | Automated interpretability aims to translate large language model (LLM) features into human understandable descriptions. However, natural language feature descriptions can be vague, inconsistent, and require manual relabeling. In response, we introduce semantic regexes, structured language descriptions of LLM features. By combining primitives that capture linguistic and semantic patterns with modifiers for contextualization, composition, and quantification, semantic regexes produce precise and expressive feature descriptions. Across quantitative benchmarks and qualitative analyses, semantic regexes match the accuracy of natural language while yielding more concise and consistent feature descriptions. Their inherent structure affords new types of analyses, including quantifying feature complexity across layers, scaling automated interpretability from insights into individual features to model-wide patterns. Finally, in user studies, we find that semantic regexes help people build accurate mental models of LLM features. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_06378 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Semantic Regexes: Auto-Interpreting LLM Features with a Structured Language Boggust, Angie Ren, Donghao Assogba, Yannick Moritz, Dominik Satyanarayan, Arvind Hohman, Fred Computation and Language Automated interpretability aims to translate large language model (LLM) features into human understandable descriptions. However, natural language feature descriptions can be vague, inconsistent, and require manual relabeling. In response, we introduce semantic regexes, structured language descriptions of LLM features. By combining primitives that capture linguistic and semantic patterns with modifiers for contextualization, composition, and quantification, semantic regexes produce precise and expressive feature descriptions. Across quantitative benchmarks and qualitative analyses, semantic regexes match the accuracy of natural language while yielding more concise and consistent feature descriptions. Their inherent structure affords new types of analyses, including quantifying feature complexity across layers, scaling automated interpretability from insights into individual features to model-wide patterns. Finally, in user studies, we find that semantic regexes help people build accurate mental models of LLM features. |
| title | Semantic Regexes: Auto-Interpreting LLM Features with a Structured Language |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2510.06378 |