Speak or Stay Silent: Context-Aware Turn-Taking in Multi-Party Dialogue
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917334643376128 |
|---|---|
| author | Bhagtani, Kratika Anand, Mrinal Xu, Yu Chen Yadav, Amit Kumar Singh |
| author_facet | Bhagtani, Kratika Anand, Mrinal Xu, Yu Chen Yadav, Amit Kumar Singh |
| contents | Existing voice AI assistants treat every detected pause as an invitation to speak. This works in dyadic dialogue, but in multi-party settings, where an AI assistant participates alongside multiple speakers, pauses are abundant and ambiguous. An assistant that speaks on every pause becomes disruptive rather than useful. In this work, we formulate context-aware turn-taking: at every detected pause, given the full conversation context, our method decides whether the assistant should speak or stay silent. We introduce a benchmark of over 120K labeled conversations spanning three multi-party corpora. Evaluating eight recent large language models, we find that they consistently fail at context-aware turn-taking under zero-shot prompting. We then propose a supervised fine-tuning approach with reasoning traces, improving balanced accuracy by up to 23 percentage points. Our findings suggest that context-aware turn-taking is not an emergent capability; it must be explicitly trained. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_11409 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Speak or Stay Silent: Context-Aware Turn-Taking in Multi-Party Dialogue Bhagtani, Kratika Anand, Mrinal Xu, Yu Chen Yadav, Amit Kumar Singh Artificial Intelligence Computation and Language Existing voice AI assistants treat every detected pause as an invitation to speak. This works in dyadic dialogue, but in multi-party settings, where an AI assistant participates alongside multiple speakers, pauses are abundant and ambiguous. An assistant that speaks on every pause becomes disruptive rather than useful. In this work, we formulate context-aware turn-taking: at every detected pause, given the full conversation context, our method decides whether the assistant should speak or stay silent. We introduce a benchmark of over 120K labeled conversations spanning three multi-party corpora. Evaluating eight recent large language models, we find that they consistently fail at context-aware turn-taking under zero-shot prompting. We then propose a supervised fine-tuning approach with reasoning traces, improving balanced accuracy by up to 23 percentage points. Our findings suggest that context-aware turn-taking is not an emergent capability; it must be explicitly trained. |
| title | Speak or Stay Silent: Context-Aware Turn-Taking in Multi-Party Dialogue |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2603.11409 |