ExpertLens: Activation steering features are highly interpretable
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909885087612928 |
|---|---|
| author | Fedzechkina, Masha Gualdoni, Eleonora Williamson, Sinead Metcalf, Katherine Seto, Skyler Theobald, Barry-John |
| author_facet | Fedzechkina, Masha Gualdoni, Eleonora Williamson, Sinead Metcalf, Katherine Seto, Skyler Theobald, Barry-John |
| contents | Activation steering methods in large language models (LLMs) have emerged as an effective way to perform targeted updates to enhance generated language without requiring large amounts of adaptation data. We ask whether the features discovered by activation steering methods are interpretable. We identify neurons responsible for specific concepts (e.g., ``cat'') using the ``finding experts'' method from research on activation steering and show that the ExpertLens, i.e., inspection of these neurons provides insights about model representation. We find that ExpertLens representations are stable across models and datasets and closely align with human representations inferred from behavioral data, matching inter-human alignment levels. ExpertLens significantly outperforms the alignment captured by word/sentence embeddings. By reconstructing human concept organization through ExpertLens, we show that it enables a granular view of LLM concept representation. Our findings suggest that ExpertLens is a flexible and lightweight approach for capturing and analyzing model representations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_15090 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ExpertLens: Activation steering features are highly interpretable Fedzechkina, Masha Gualdoni, Eleonora Williamson, Sinead Metcalf, Katherine Seto, Skyler Theobald, Barry-John Computation and Language Artificial Intelligence Activation steering methods in large language models (LLMs) have emerged as an effective way to perform targeted updates to enhance generated language without requiring large amounts of adaptation data. We ask whether the features discovered by activation steering methods are interpretable. We identify neurons responsible for specific concepts (e.g., ``cat'') using the ``finding experts'' method from research on activation steering and show that the ExpertLens, i.e., inspection of these neurons provides insights about model representation. We find that ExpertLens representations are stable across models and datasets and closely align with human representations inferred from behavioral data, matching inter-human alignment levels. ExpertLens significantly outperforms the alignment captured by word/sentence embeddings. By reconstructing human concept organization through ExpertLens, we show that it enables a granular view of LLM concept representation. Our findings suggest that ExpertLens is a flexible and lightweight approach for capturing and analyzing model representations. |
| title | ExpertLens: Activation steering features are highly interpretable |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2502.15090 |