Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2403.14705 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909145284739072 |
|---|---|
| author | Carmeli, Boaz Belinkov, Yonatan Meir, Ron |
| author_facet | Carmeli, Boaz Belinkov, Yonatan Meir, Ron |
| contents | Artificial agents that learn to communicate in order to accomplish a given task acquire communication protocols that are typically opaque to a human. A large body of work has attempted to evaluate the emergent communication via various evaluation measures, with \emph{compositionality} featuring as a prominent desired trait. However, current evaluation procedures do not directly expose the compositionality of the emergent communication. We propose a procedure to assess the compositionality of emergent communication by finding the best-match between emerged words and natural language concepts. The best-match algorithm provides both a global score and a translation-map from emergent words to natural language concepts. To the best of our knowledge, it is the first time that such direct and interpretable mapping between emergent words and human concepts is provided. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_14705 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Concept-Best-Matching: Evaluating Compositionality in Emergent Communication Carmeli, Boaz Belinkov, Yonatan Meir, Ron Artificial Intelligence Computation and Language Artificial agents that learn to communicate in order to accomplish a given task acquire communication protocols that are typically opaque to a human. A large body of work has attempted to evaluate the emergent communication via various evaluation measures, with \emph{compositionality} featuring as a prominent desired trait. However, current evaluation procedures do not directly expose the compositionality of the emergent communication. We propose a procedure to assess the compositionality of emergent communication by finding the best-match between emerged words and natural language concepts. The best-match algorithm provides both a global score and a translation-map from emergent words to natural language concepts. To the best of our knowledge, it is the first time that such direct and interpretable mapping between emergent words and human concepts is provided. |
| title | Concept-Best-Matching: Evaluating Compositionality in Emergent Communication |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2403.14705 |