Catalytic Role Of Noise And Necessity Of Inductive Biases In The Emergence Of Compositional Communication
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2021
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913295935471616 |
|---|---|
| author | Kuciński, Łukasz Korbak, Tomasz Kołodziej, Paweł Miłoś, Piotr |
| author_facet | Kuciński, Łukasz Korbak, Tomasz Kołodziej, Paweł Miłoś, Piotr |
| contents | Communication is compositional if complex signals can be represented as a combination of simpler subparts. In this paper, we theoretically show that inductive biases on both the training framework and the data are needed to develop a compositional communication. Moreover, we prove that compositionality spontaneously arises in the signaling games, where agents communicate over a noisy channel. We experimentally confirm that a range of noise levels, which depends on the model and the data, indeed promotes compositionality. Finally, we provide a comprehensive study of this dependence and report results in terms of recently studied compositionality metrics: topographical similarity, conflict count, and context independence. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2111_06464 |
| institution | arXiv |
| publishDate | 2021 |
| record_format | arxiv |
| spellingShingle | Catalytic Role Of Noise And Necessity Of Inductive Biases In The Emergence Of Compositional Communication Kuciński, Łukasz Korbak, Tomasz Kołodziej, Paweł Miłoś, Piotr Machine Learning Artificial Intelligence Computation and Language Communication is compositional if complex signals can be represented as a combination of simpler subparts. In this paper, we theoretically show that inductive biases on both the training framework and the data are needed to develop a compositional communication. Moreover, we prove that compositionality spontaneously arises in the signaling games, where agents communicate over a noisy channel. We experimentally confirm that a range of noise levels, which depends on the model and the data, indeed promotes compositionality. Finally, we provide a comprehensive study of this dependence and report results in terms of recently studied compositionality metrics: topographical similarity, conflict count, and context independence. |
| title | Catalytic Role Of Noise And Necessity Of Inductive Biases In The Emergence Of Compositional Communication |
| topic | Machine Learning Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2111.06464 |