Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2407.14662 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909262973763584 |
|---|---|
| author | Wattenberg, Martin Viégas, Fernanda B. |
| author_facet | Wattenberg, Martin Viégas, Fernanda B. |
| contents | Many neural nets appear to represent data as linear combinations of "feature vectors." Algorithms for discovering these vectors have seen impressive recent success. However, we argue that this success is incomplete without an understanding of relational composition: how (or whether) neural nets combine feature vectors to represent more complicated relationships. To facilitate research in this area, this paper offers a guided tour of various relational mechanisms that have been proposed, along with preliminary analysis of how such mechanisms might affect the search for interpretable features. We end with a series of promising areas for empirical research, which may help determine how neural networks represent structured data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_14662 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Relational Composition in Neural Networks: A Survey and Call to Action Wattenberg, Martin Viégas, Fernanda B. Artificial Intelligence Machine Learning Many neural nets appear to represent data as linear combinations of "feature vectors." Algorithms for discovering these vectors have seen impressive recent success. However, we argue that this success is incomplete without an understanding of relational composition: how (or whether) neural nets combine feature vectors to represent more complicated relationships. To facilitate research in this area, this paper offers a guided tour of various relational mechanisms that have been proposed, along with preliminary analysis of how such mechanisms might affect the search for interpretable features. We end with a series of promising areas for empirical research, which may help determine how neural networks represent structured data. |
| title | Relational Composition in Neural Networks: A Survey and Call to Action |
| topic | Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2407.14662 |