Enhancing Speech Instruction Understanding and Disambiguation in Robotics via Speech Prosody
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913871956017152 |
|---|---|
| author | Sasu, David Yamoah, Kweku Andoh Quartey, Benedict Schluter, Natalie |
| author_facet | Sasu, David Yamoah, Kweku Andoh Quartey, Benedict Schluter, Natalie |
| contents | Enabling robots to accurately interpret and execute spoken language instructions is essential for effective human-robot collaboration. Traditional methods rely on speech recognition to transcribe speech into text, often discarding crucial prosodic cues needed for disambiguating intent. We propose a novel approach that directly leverages speech prosody to infer and resolve instruction intent. Predicted intents are integrated into large language models via in-context learning to disambiguate and select appropriate task plans. Additionally, we present the first ambiguous speech dataset for robotics, designed to advance research in speech disambiguation. Our method achieves 95.79% accuracy in detecting referent intents within an utterance and determines the intended task plan of ambiguous instructions with 71.96% accuracy, demonstrating its potential to significantly improve human-robot communication. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_02057 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Enhancing Speech Instruction Understanding and Disambiguation in Robotics via Speech Prosody Sasu, David Yamoah, Kweku Andoh Quartey, Benedict Schluter, Natalie Robotics Artificial Intelligence Computation and Language Machine Learning Enabling robots to accurately interpret and execute spoken language instructions is essential for effective human-robot collaboration. Traditional methods rely on speech recognition to transcribe speech into text, often discarding crucial prosodic cues needed for disambiguating intent. We propose a novel approach that directly leverages speech prosody to infer and resolve instruction intent. Predicted intents are integrated into large language models via in-context learning to disambiguate and select appropriate task plans. Additionally, we present the first ambiguous speech dataset for robotics, designed to advance research in speech disambiguation. Our method achieves 95.79% accuracy in detecting referent intents within an utterance and determines the intended task plan of ambiguous instructions with 71.96% accuracy, demonstrating its potential to significantly improve human-robot communication. |
| title | Enhancing Speech Instruction Understanding and Disambiguation in Robotics via Speech Prosody |
| topic | Robotics Artificial Intelligence Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2506.02057 |