Understanding Task Representations in Neural Networks via Bayesian Ablation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914443939545088 |
|---|---|
| author | Nam, Andrew Campbell, Declan Griffiths, Thomas Cohen, Jonathan Leslie, Sarah-Jane |
| author_facet | Nam, Andrew Campbell, Declan Griffiths, Thomas Cohen, Jonathan Leslie, Sarah-Jane |
| contents | Neural networks are powerful tools for cognitive modeling due to their flexibility and emergent properties. However, interpreting their learned representations remains challenging due to their sub-symbolic semantics. In this work, we introduce a novel probabilistic framework for interpreting latent task representations in neural networks. Inspired by Bayesian inference, our approach defines a distribution over representational units to infer their causal contributions to task performance. Using ideas from information theory, we propose a suite of tools and metrics to illuminate key model properties, including representational distributedness, manifold complexity, and polysemanticity. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_13742 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Understanding Task Representations in Neural Networks via Bayesian Ablation Nam, Andrew Campbell, Declan Griffiths, Thomas Cohen, Jonathan Leslie, Sarah-Jane Machine Learning Artificial Intelligence Neural networks are powerful tools for cognitive modeling due to their flexibility and emergent properties. However, interpreting their learned representations remains challenging due to their sub-symbolic semantics. In this work, we introduce a novel probabilistic framework for interpreting latent task representations in neural networks. Inspired by Bayesian inference, our approach defines a distribution over representational units to infer their causal contributions to task performance. Using ideas from information theory, we propose a suite of tools and metrics to illuminate key model properties, including representational distributedness, manifold complexity, and polysemanticity. |
| title | Understanding Task Representations in Neural Networks via Bayesian Ablation |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2505.13742 |