Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2502.16994 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917431823302656 |
|---|---|
| author | Puri, Bruno Jain, Aakriti Golimblevskaia, Elena Kahardipraja, Patrick Wiegand, Thomas Samek, Wojciech Lapuschkin, Sebastian |
| author_facet | Puri, Bruno Jain, Aakriti Golimblevskaia, Elena Kahardipraja, Patrick Wiegand, Thomas Samek, Wojciech Lapuschkin, Sebastian |
| contents | Recent advances in mechanistic interpretability have highlighted the potential of automating interpretability pipelines in analyzing the latent representations within LLMs. While this may enhance our understanding of internal mechanisms, the field lacks standardized evaluation methods for assessing the validity of discovered features. We attempt to bridge this gap by introducing FADE: Feature Alignment to Description Evaluation, a scalable model-agnostic framework for automatically evaluating feature-to-description alignment. FADE evaluates alignment across four key metrics - Clarity, Responsiveness, Purity, and Faithfulness - and systematically quantifies the causes of the misalignment between features and their descriptions. We apply FADE to analyze existing open-source feature descriptions and assess key components of automated interpretability pipelines, aiming to enhance the quality of descriptions. Our findings highlight fundamental challenges in generating feature descriptions, particularly for SAEs compared to MLP neurons, providing insights into the limitations and future directions of automated interpretability. We release FADE as an open-source package at: https://github.com/brunibrun/FADE |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_16994 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | FADE: Why Bad Descriptions Happen to Good Features Puri, Bruno Jain, Aakriti Golimblevskaia, Elena Kahardipraja, Patrick Wiegand, Thomas Samek, Wojciech Lapuschkin, Sebastian Machine Learning Artificial Intelligence Computation and Language Recent advances in mechanistic interpretability have highlighted the potential of automating interpretability pipelines in analyzing the latent representations within LLMs. While this may enhance our understanding of internal mechanisms, the field lacks standardized evaluation methods for assessing the validity of discovered features. We attempt to bridge this gap by introducing FADE: Feature Alignment to Description Evaluation, a scalable model-agnostic framework for automatically evaluating feature-to-description alignment. FADE evaluates alignment across four key metrics - Clarity, Responsiveness, Purity, and Faithfulness - and systematically quantifies the causes of the misalignment between features and their descriptions. We apply FADE to analyze existing open-source feature descriptions and assess key components of automated interpretability pipelines, aiming to enhance the quality of descriptions. Our findings highlight fundamental challenges in generating feature descriptions, particularly for SAEs compared to MLP neurons, providing insights into the limitations and future directions of automated interpretability. We release FADE as an open-source package at: https://github.com/brunibrun/FADE |
| title | FADE: Why Bad Descriptions Happen to Good Features |
| topic | Machine Learning Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2502.16994 |