Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2309.11737 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911799991861248 |
|---|---|
| author | Hou, Zhaoyi Joey Zhang, Li Callison-Burch, Chris |
| author_facet | Hou, Zhaoyi Joey Zhang, Li Callison-Burch, Chris |
| contents | Script learning studies how stereotypical events unfold, enabling machines to reason about narratives with implicit information. Previous works mostly consider a script as a linear sequence of events while ignoring the potential branches that arise due to people's circumstantial choices. We hence propose Choice-75, the first benchmark that challenges intelligent systems to make decisions given descriptive scenarios, containing 75 scripts and more than 600 scenarios. We also present preliminary results with current large language models (LLM). Although they demonstrate overall decent performance, there is still notable headroom in hard scenarios. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_11737 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Choice-75: A Dataset on Decision Branching in Script Learning Hou, Zhaoyi Joey Zhang, Li Callison-Burch, Chris Artificial Intelligence Script learning studies how stereotypical events unfold, enabling machines to reason about narratives with implicit information. Previous works mostly consider a script as a linear sequence of events while ignoring the potential branches that arise due to people's circumstantial choices. We hence propose Choice-75, the first benchmark that challenges intelligent systems to make decisions given descriptive scenarios, containing 75 scripts and more than 600 scenarios. We also present preliminary results with current large language models (LLM). Although they demonstrate overall decent performance, there is still notable headroom in hard scenarios. |
| title | Choice-75: A Dataset on Decision Branching in Script Learning |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2309.11737 |