Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2310.16316 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908575701401600 |
|---|---|
| author | You, Weiqiu Qu, Helen Gatti, Marco Jain, Bhuvnesh Wong, Eric |
| author_facet | You, Weiqiu Qu, Helen Gatti, Marco Jain, Bhuvnesh Wong, Eric |
| contents | Self-attributing neural networks (SANNs) present a potential path towards interpretable models for high-dimensional problems, but often face significant trade-offs in performance. In this work, we formally prove a lower bound on errors of per-feature SANNs, whereas group-based SANNs can achieve zero error and thus high performance. Motivated by these insights, we propose Sum-of-Parts (SOP), a framework that transforms any differentiable model into a group-based SANN, where feature groups are learned end-to-end without group supervision. SOP achieves state-of-the-art performance for SANNs on vision and language tasks, and we validate that the groups are interpretable on a range of quantitative and semantic metrics. We further validate the utility of SOP explanations in model debugging and cosmological scientific discovery. Our code is available at https://github.com/BrachioLab/sop |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_16316 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Sum-of-Parts: Self-Attributing Neural Networks with End-to-End Learning of Feature Groups You, Weiqiu Qu, Helen Gatti, Marco Jain, Bhuvnesh Wong, Eric Machine Learning Artificial Intelligence Self-attributing neural networks (SANNs) present a potential path towards interpretable models for high-dimensional problems, but often face significant trade-offs in performance. In this work, we formally prove a lower bound on errors of per-feature SANNs, whereas group-based SANNs can achieve zero error and thus high performance. Motivated by these insights, we propose Sum-of-Parts (SOP), a framework that transforms any differentiable model into a group-based SANN, where feature groups are learned end-to-end without group supervision. SOP achieves state-of-the-art performance for SANNs on vision and language tasks, and we validate that the groups are interpretable on a range of quantitative and semantic metrics. We further validate the utility of SOP explanations in model debugging and cosmological scientific discovery. Our code is available at https://github.com/BrachioLab/sop |
| title | Sum-of-Parts: Self-Attributing Neural Networks with End-to-End Learning of Feature Groups |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2310.16316 |