When Are Two Networks the Same? Tensor Similarity for Mechanistic Interpretability
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914567706116096 |
|---|---|
| author | Gonzalez, ML Nissen Albuquerque, Melwina Wroe, Laurence Cohen, Jacob Meyer Smith, Logan Riggs Dooms, Thomas |
| author_facet | Gonzalez, ML Nissen Albuquerque, Melwina Wroe, Laurence Cohen, Jacob Meyer Smith, Logan Riggs Dooms, Thomas |
| contents | Mechanistic interpretability aims to break models into meaningful parts; verifying that two such parts implement the same computation is a prerequisite. Existing similarity measures evaluate either empirical behaviour, leaving them blind to out-of-distribution mechanisms, or basis-dependent parameters, meaning they disregard weight-space symmetries. To address these issues for the class of tensor-based models, we introduce a weight-based metric, tensor similarity, that is invariant to such symmetries. This metric captures global functional equivalence and accounts for cross-layer mechanisms using an efficient recursive algorithm. Empirically, tensor similarity tracks functional training dynamics, such as grokking and backdoor insertion, with higher fidelity than existing metrics. This reduces measuring similarity and verifying faithfulness into a solved algebraic problem rather than one of empirical approximation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_15183 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | When Are Two Networks the Same? Tensor Similarity for Mechanistic Interpretability Gonzalez, ML Nissen Albuquerque, Melwina Wroe, Laurence Cohen, Jacob Meyer Smith, Logan Riggs Dooms, Thomas Machine Learning Mechanistic interpretability aims to break models into meaningful parts; verifying that two such parts implement the same computation is a prerequisite. Existing similarity measures evaluate either empirical behaviour, leaving them blind to out-of-distribution mechanisms, or basis-dependent parameters, meaning they disregard weight-space symmetries. To address these issues for the class of tensor-based models, we introduce a weight-based metric, tensor similarity, that is invariant to such symmetries. This metric captures global functional equivalence and accounts for cross-layer mechanisms using an efficient recursive algorithm. Empirically, tensor similarity tracks functional training dynamics, such as grokking and backdoor insertion, with higher fidelity than existing metrics. This reduces measuring similarity and verifying faithfulness into a solved algebraic problem rather than one of empirical approximation. |
| title | When Are Two Networks the Same? Tensor Similarity for Mechanistic Interpretability |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2605.15183 |