Behavioural vs. Representational Systematicity in End-to-End Models: An Opinionated Survey
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909638820102144 |
|---|---|
| author | Vegner, Ivan de Souza, Sydelle Forch, Valentin Lewis, Martha Doumas, Leonidas A. A. |
| author_facet | Vegner, Ivan de Souza, Sydelle Forch, Valentin Lewis, Martha Doumas, Leonidas A. A. |
| contents | A core aspect of compositionality, systematicity is a desirable property in ML models as it enables strong generalization to novel contexts. This has led to numerous studies proposing benchmarks to assess systematic generalization, as well as models and training regimes designed to enhance it. Many of these efforts are framed as addressing the challenge posed by Fodor and Pylyshyn. However, while they argue for systematicity of representations, existing benchmarks and models primarily focus on the systematicity of behaviour. We emphasize the crucial nature of this distinction. Furthermore, building on Hadley's (1994) taxonomy of systematic generalization, we analyze the extent to which behavioural systematicity is tested by key benchmarks in the literature across language and vision. Finally, we highlight ways of assessing systematicity of representations in ML models as practiced in the field of mechanistic interpretability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_04461 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Behavioural vs. Representational Systematicity in End-to-End Models: An Opinionated Survey Vegner, Ivan de Souza, Sydelle Forch, Valentin Lewis, Martha Doumas, Leonidas A. A. Machine Learning Artificial Intelligence Computation and Language I.2.6; I.2.0; I.2.7 A core aspect of compositionality, systematicity is a desirable property in ML models as it enables strong generalization to novel contexts. This has led to numerous studies proposing benchmarks to assess systematic generalization, as well as models and training regimes designed to enhance it. Many of these efforts are framed as addressing the challenge posed by Fodor and Pylyshyn. However, while they argue for systematicity of representations, existing benchmarks and models primarily focus on the systematicity of behaviour. We emphasize the crucial nature of this distinction. Furthermore, building on Hadley's (1994) taxonomy of systematic generalization, we analyze the extent to which behavioural systematicity is tested by key benchmarks in the literature across language and vision. Finally, we highlight ways of assessing systematicity of representations in ML models as practiced in the field of mechanistic interpretability. |
| title | Behavioural vs. Representational Systematicity in End-to-End Models: An Opinionated Survey |
| topic | Machine Learning Artificial Intelligence Computation and Language I.2.6; I.2.0; I.2.7 |
| url | https://arxiv.org/abs/2506.04461 |