PURE: Turning Polysemantic Neurons Into Pure Features by Identifying Relevant Circuits
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914746398146560 |
|---|---|
| author | Dreyer, Maximilian Purelku, Erblina Vielhaben, Johanna Samek, Wojciech Lapuschkin, Sebastian |
| author_facet | Dreyer, Maximilian Purelku, Erblina Vielhaben, Johanna Samek, Wojciech Lapuschkin, Sebastian |
| contents | The field of mechanistic interpretability aims to study the role of individual neurons in Deep Neural Networks. Single neurons, however, have the capability to act polysemantically and encode for multiple (unrelated) features, which renders their interpretation difficult. We present a method for disentangling polysemanticity of any Deep Neural Network by decomposing a polysemantic neuron into multiple monosemantic "virtual" neurons. This is achieved by identifying the relevant sub-graph ("circuit") for each "pure" feature. We demonstrate how our approach allows us to find and disentangle various polysemantic units of ResNet models trained on ImageNet. While evaluating feature visualizations using CLIP, our method effectively disentangles representations, improving upon methods based on neuron activations. Our code is available at https://github.com/maxdreyer/PURE. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_06453 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | PURE: Turning Polysemantic Neurons Into Pure Features by Identifying Relevant Circuits Dreyer, Maximilian Purelku, Erblina Vielhaben, Johanna Samek, Wojciech Lapuschkin, Sebastian Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning The field of mechanistic interpretability aims to study the role of individual neurons in Deep Neural Networks. Single neurons, however, have the capability to act polysemantically and encode for multiple (unrelated) features, which renders their interpretation difficult. We present a method for disentangling polysemanticity of any Deep Neural Network by decomposing a polysemantic neuron into multiple monosemantic "virtual" neurons. This is achieved by identifying the relevant sub-graph ("circuit") for each "pure" feature. We demonstrate how our approach allows us to find and disentangle various polysemantic units of ResNet models trained on ImageNet. While evaluating feature visualizations using CLIP, our method effectively disentangles representations, improving upon methods based on neuron activations. Our code is available at https://github.com/maxdreyer/PURE. |
| title | PURE: Turning Polysemantic Neurons Into Pure Features by Identifying Relevant Circuits |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2404.06453 |