Compositional Structures in Neural Embedding and Interaction Decompositions
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916320367345664 |
|---|---|
| author | Trager, Matthew Achille, Alessandro Perera, Pramuditha Zancato, Luca Soatto, Stefano |
| author_facet | Trager, Matthew Achille, Alessandro Perera, Pramuditha Zancato, Luca Soatto, Stefano |
| contents | We describe a basic correspondence between linear algebraic structures within vector embeddings in artificial neural networks and conditional independence constraints on the probability distributions modeled by these networks. Our framework aims to shed light on the emergence of structural patterns in data representations, a phenomenon widely acknowledged but arguably still lacking a solid formal grounding. Specifically, we introduce a characterization of compositional structures in terms of "interaction decompositions," and we establish necessary and sufficient conditions for the presence of such structures within the representations of a model. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_08934 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Compositional Structures in Neural Embedding and Interaction Decompositions Trager, Matthew Achille, Alessandro Perera, Pramuditha Zancato, Luca Soatto, Stefano Machine Learning We describe a basic correspondence between linear algebraic structures within vector embeddings in artificial neural networks and conditional independence constraints on the probability distributions modeled by these networks. Our framework aims to shed light on the emergence of structural patterns in data representations, a phenomenon widely acknowledged but arguably still lacking a solid formal grounding. Specifically, we introduce a characterization of compositional structures in terms of "interaction decompositions," and we establish necessary and sufficient conditions for the presence of such structures within the representations of a model. |
| title | Compositional Structures in Neural Embedding and Interaction Decompositions |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2407.08934 |