Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2407.13594 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909654881140736 |
|---|---|
| author | Palumbo, Nils Mangal, Ravi Wang, Zifan Vijayakumar, Saranya Pasareanu, Corina S. Jha, Somesh |
| author_facet | Palumbo, Nils Mangal, Ravi Wang, Zifan Vijayakumar, Saranya Pasareanu, Corina S. Jha, Somesh |
| contents | Mechanistic interpretability aims to reverse engineer the computation performed by a neural network in terms of its internal components. Although there is a growing body of research on mechanistic interpretation of neural networks, the notion of a mechanistic interpretation itself is often ad-hoc. Inspired by the notion of abstract interpretation from the program analysis literature that aims to develop approximate semantics for programs, we give a set of axioms that formally characterize a mechanistic interpretation as a description that approximately captures the semantics of the neural network under analysis in a compositional manner. We demonstrate the applicability of these axioms for validating mechanistic interpretations on an existing, well-known interpretability study as well as on a new case study involving a Transformer-based model trained to solve the well-known 2-SAT problem. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_13594 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Validating Mechanistic Interpretations: An Axiomatic Approach Palumbo, Nils Mangal, Ravi Wang, Zifan Vijayakumar, Saranya Pasareanu, Corina S. Jha, Somesh Machine Learning Mechanistic interpretability aims to reverse engineer the computation performed by a neural network in terms of its internal components. Although there is a growing body of research on mechanistic interpretation of neural networks, the notion of a mechanistic interpretation itself is often ad-hoc. Inspired by the notion of abstract interpretation from the program analysis literature that aims to develop approximate semantics for programs, we give a set of axioms that formally characterize a mechanistic interpretation as a description that approximately captures the semantics of the neural network under analysis in a compositional manner. We demonstrate the applicability of these axioms for validating mechanistic interpretations on an existing, well-known interpretability study as well as on a new case study involving a Transformer-based model trained to solve the well-known 2-SAT problem. |
| title | Validating Mechanistic Interpretations: An Axiomatic Approach |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2407.13594 |