Nebula: A discourse aware Minecraft Builder
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909341595992064 |
|---|---|
| author | Chaturvedi, Akshay Thompson, Kate Asher, Nicholas |
| author_facet | Chaturvedi, Akshay Thompson, Kate Asher, Nicholas |
| contents | When engaging in collaborative tasks, humans efficiently exploit the semantic structure of a conversation to optimize verbal and nonverbal interactions. But in recent "language to code" or "language to action" models, this information is lacking. We show how incorporating the prior discourse and nonlinguistic context of a conversation situated in a nonlinguistic environment can improve the "language to action" component of such interactions. We finetune an LLM to predict actions based on prior context; our model, Nebula, doubles the net-action F1 score over the baseline on this task of Jayannavar et al.(2020). We also investigate our model's ability to construct shapes and understand location descriptions using a synthetic dataset |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_18164 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Nebula: A discourse aware Minecraft Builder Chaturvedi, Akshay Thompson, Kate Asher, Nicholas Computation and Language Machine Learning When engaging in collaborative tasks, humans efficiently exploit the semantic structure of a conversation to optimize verbal and nonverbal interactions. But in recent "language to code" or "language to action" models, this information is lacking. We show how incorporating the prior discourse and nonlinguistic context of a conversation situated in a nonlinguistic environment can improve the "language to action" component of such interactions. We finetune an LLM to predict actions based on prior context; our model, Nebula, doubles the net-action F1 score over the baseline on this task of Jayannavar et al.(2020). We also investigate our model's ability to construct shapes and understand location descriptions using a synthetic dataset |
| title | Nebula: A discourse aware Minecraft Builder |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2406.18164 |