GenComUI: Exploring Generative Visual Aids as Medium to Support Task-Oriented Human-Robot Communication
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913692247916544 |
|---|---|
| author | Ge, Yate Li, Meiying Huang, Xipeng Hu, Yuanda Wang, Qi Sun, Xiaohua Guo, Weiwei |
| author_facet | Ge, Yate Li, Meiying Huang, Xipeng Hu, Yuanda Wang, Qi Sun, Xiaohua Guo, Weiwei |
| contents | This work investigates the integration of generative visual aids in human-robot task communication. We developed GenComUI, a system powered by large language models that dynamically generates contextual visual aids (such as map annotations, path indicators, and animations) to support verbal task communication and facilitate the generation of customized task programs for the robot. This system was informed by a formative study that examined how humans use external visual tools to assist verbal communication in spatial tasks. To evaluate its effectiveness, we conducted a user experiment (n = 20) comparing GenComUI with a voice-only baseline. The results demonstrate that generative visual aids, through both qualitative and quantitative analysis, enhance verbal task communication by providing continuous visual feedback, thus promoting natural and effective human-robot communication. Additionally, the study offers a set of design implications, emphasizing how dynamically generated visual aids can serve as an effective communication medium in human-robot interaction. These findings underscore the potential of generative visual aids to inform the design of more intuitive and effective human-robot communication, particularly for complex communication scenarios in human-robot interaction and LLM-based end-user development. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_10678 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | GenComUI: Exploring Generative Visual Aids as Medium to Support Task-Oriented Human-Robot Communication Ge, Yate Li, Meiying Huang, Xipeng Hu, Yuanda Wang, Qi Sun, Xiaohua Guo, Weiwei Human-Computer Interaction Artificial Intelligence Robotics H.5.2; H.5.3; I.2.7; I.2.0 This work investigates the integration of generative visual aids in human-robot task communication. We developed GenComUI, a system powered by large language models that dynamically generates contextual visual aids (such as map annotations, path indicators, and animations) to support verbal task communication and facilitate the generation of customized task programs for the robot. This system was informed by a formative study that examined how humans use external visual tools to assist verbal communication in spatial tasks. To evaluate its effectiveness, we conducted a user experiment (n = 20) comparing GenComUI with a voice-only baseline. The results demonstrate that generative visual aids, through both qualitative and quantitative analysis, enhance verbal task communication by providing continuous visual feedback, thus promoting natural and effective human-robot communication. Additionally, the study offers a set of design implications, emphasizing how dynamically generated visual aids can serve as an effective communication medium in human-robot interaction. These findings underscore the potential of generative visual aids to inform the design of more intuitive and effective human-robot communication, particularly for complex communication scenarios in human-robot interaction and LLM-based end-user development. |
| title | GenComUI: Exploring Generative Visual Aids as Medium to Support Task-Oriented Human-Robot Communication |
| topic | Human-Computer Interaction Artificial Intelligence Robotics H.5.2; H.5.3; I.2.7; I.2.0 |
| url | https://arxiv.org/abs/2502.10678 |