G-VOILA: Gaze-Facilitated Information Querying in Daily Scenarios
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914794065362944 |
|---|---|
| author | Wang, Zeyu Shi, Yuanchun Wang, Yuntao Yao, Yuchen Yan, Kun Wang, Yuhan Ji, Lei Xu, Xuhai Yu, Chun |
| author_facet | Wang, Zeyu Shi, Yuanchun Wang, Yuntao Yao, Yuchen Yan, Kun Wang, Yuhan Ji, Lei Xu, Xuhai Yu, Chun |
| contents | Modern information querying systems are progressively incorporating multimodal inputs like vision and audio. However, the integration of gaze -- a modality deeply linked to user intent and increasingly accessible via gaze-tracking wearables -- remains underexplored. This paper introduces a novel gaze-facilitated information querying paradigm, named G-VOILA, which synergizes users' gaze, visual field, and voice-based natural language queries to facilitate a more intuitive querying process. In a user-enactment study involving 21 participants in 3 daily scenarios (p = 21, scene = 3), we revealed the ambiguity in users' query language and a gaze-voice coordination pattern in users' natural query behaviors with G-VOILA. Based on the quantitative and qualitative findings, we developed a design framework for the G-VOILA paradigm, which effectively integrates the gaze data with the in-situ querying context. Then we implemented a G-VOILA proof-of-concept using cutting-edge deep learning techniques. A follow-up user study (p = 16, scene = 2) demonstrates its effectiveness by achieving both higher objective score and subjective score, compared to a baseline without gaze data. We further conducted interviews and provided insights for future gaze-facilitated information querying systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_07652 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | G-VOILA: Gaze-Facilitated Information Querying in Daily Scenarios Wang, Zeyu Shi, Yuanchun Wang, Yuntao Yao, Yuchen Yan, Kun Wang, Yuhan Ji, Lei Xu, Xuhai Yu, Chun Human-Computer Interaction Artificial Intelligence Modern information querying systems are progressively incorporating multimodal inputs like vision and audio. However, the integration of gaze -- a modality deeply linked to user intent and increasingly accessible via gaze-tracking wearables -- remains underexplored. This paper introduces a novel gaze-facilitated information querying paradigm, named G-VOILA, which synergizes users' gaze, visual field, and voice-based natural language queries to facilitate a more intuitive querying process. In a user-enactment study involving 21 participants in 3 daily scenarios (p = 21, scene = 3), we revealed the ambiguity in users' query language and a gaze-voice coordination pattern in users' natural query behaviors with G-VOILA. Based on the quantitative and qualitative findings, we developed a design framework for the G-VOILA paradigm, which effectively integrates the gaze data with the in-situ querying context. Then we implemented a G-VOILA proof-of-concept using cutting-edge deep learning techniques. A follow-up user study (p = 16, scene = 2) demonstrates its effectiveness by achieving both higher objective score and subjective score, compared to a baseline without gaze data. We further conducted interviews and provided insights for future gaze-facilitated information querying systems. |
| title | G-VOILA: Gaze-Facilitated Information Querying in Daily Scenarios |
| topic | Human-Computer Interaction Artificial Intelligence |
| url | https://arxiv.org/abs/2405.07652 |