Reference Games as a Testbed for the Alignment of Model Uncertainty and Clarification Requests
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913131444305920 |
|---|---|
| author | Ali, Manar Sieker, Judith Zarrieß, Sina Buschmeier, Hendrik |
| author_facet | Ali, Manar Sieker, Judith Zarrieß, Sina Buschmeier, Hendrik |
| contents | In human conversation, both interlocutors play an active role in maintaining mutual understanding. When listeners are uncertain about what speakers mean, for example, they can request clarification. It is an open question for language models whether they can assume a similar listener role, recognizing and expressing their own uncertainty through clarification. We argue that reference games are a suitable testbed to approach this question as they are controlled, self-contained, and make clarification needs explicit and measurable. To test this, we evaluate three vision-language models comparing a baseline reference resolution task to an experiment where the models are instructed to request clarification when uncertain. The results suggest that even in such simple tasks, models often struggle to recognize internal uncertainty and translate it into adequate clarification behavior. This demonstrates the value of reference games as testbeds for interaction qualities of (vision and) language models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_07820 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Reference Games as a Testbed for the Alignment of Model Uncertainty and Clarification Requests Ali, Manar Sieker, Judith Zarrieß, Sina Buschmeier, Hendrik Computation and Language In human conversation, both interlocutors play an active role in maintaining mutual understanding. When listeners are uncertain about what speakers mean, for example, they can request clarification. It is an open question for language models whether they can assume a similar listener role, recognizing and expressing their own uncertainty through clarification. We argue that reference games are a suitable testbed to approach this question as they are controlled, self-contained, and make clarification needs explicit and measurable. To test this, we evaluate three vision-language models comparing a baseline reference resolution task to an experiment where the models are instructed to request clarification when uncertain. The results suggest that even in such simple tasks, models often struggle to recognize internal uncertainty and translate it into adequate clarification behavior. This demonstrates the value of reference games as testbeds for interaction qualities of (vision and) language models. |
| title | Reference Games as a Testbed for the Alignment of Model Uncertainty and Clarification Requests |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2601.07820 |