Still Between Us? Evaluating and Improving Voice Assistant Robustness to Third-Party Interruptions
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917420394872832 |
|---|---|
| author | Lee, Dongwook Song, Eunwoo Lee, Che Hyun Kim, Heeseung Yoon, Sungroh |
| author_facet | Lee, Dongwook Song, Eunwoo Lee, Che Hyun Kim, Heeseung Yoon, Sungroh |
| contents | While recent Spoken Language Models (SLMs) have been actively deployed in real-world scenarios, they lack the capability to discern Third-Party Interruptions (TPI) from the primary user's ongoing flow, leaving them vulnerable to contextual failures. To bridge this gap, we introduce TPI-Train, a dataset of 88K instances designed with speaker-aware hard negatives to enforce acoustic cue prioritization for interruption handling, and TPI-Bench, a comprehensive evaluation framework designed to rigorously measure the interruption-handling strategy and precise speaker discrimination in deceptive contexts. Experiments demonstrate that our dataset design mitigates semantic shortcut learning-a critical pitfall where models exploit semantic context while neglecting acoustic signals essential for discerning speaker changes. We believe our work establishes a foundational resource for overcoming text-dominated unimodal reliance in SLMs, paving the way for more robust multi-party spoken interaction. The code for the framework is publicly available at https://tpi-va.github.io |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_17358 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Still Between Us? Evaluating and Improving Voice Assistant Robustness to Third-Party Interruptions Lee, Dongwook Song, Eunwoo Lee, Che Hyun Kim, Heeseung Yoon, Sungroh Computation and Language Artificial Intelligence Sound While recent Spoken Language Models (SLMs) have been actively deployed in real-world scenarios, they lack the capability to discern Third-Party Interruptions (TPI) from the primary user's ongoing flow, leaving them vulnerable to contextual failures. To bridge this gap, we introduce TPI-Train, a dataset of 88K instances designed with speaker-aware hard negatives to enforce acoustic cue prioritization for interruption handling, and TPI-Bench, a comprehensive evaluation framework designed to rigorously measure the interruption-handling strategy and precise speaker discrimination in deceptive contexts. Experiments demonstrate that our dataset design mitigates semantic shortcut learning-a critical pitfall where models exploit semantic context while neglecting acoustic signals essential for discerning speaker changes. We believe our work establishes a foundational resource for overcoming text-dominated unimodal reliance in SLMs, paving the way for more robust multi-party spoken interaction. The code for the framework is publicly available at https://tpi-va.github.io |
| title | Still Between Us? Evaluating and Improving Voice Assistant Robustness to Third-Party Interruptions |
| topic | Computation and Language Artificial Intelligence Sound |
| url | https://arxiv.org/abs/2604.17358 |