Explainable Interfaces for Rapid Gaze-Based Interactions in Mixed Reality
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910417428676608 |
|---|---|
| author | Yu, Mengjie Harris, Dustin Jones, Ian Zhang, Ting Liu, Yue Sendhilnathan, Naveen Kokhlikyan, Narine Wang, Fulton Tran, Co Livingston, Jordan L. Taylor, Krista E. Hu, Zhenhong Hood, Mary A. Benko, Hrvoje Jonker, Tanya R. |
| author_facet | Yu, Mengjie Harris, Dustin Jones, Ian Zhang, Ting Liu, Yue Sendhilnathan, Naveen Kokhlikyan, Narine Wang, Fulton Tran, Co Livingston, Jordan L. Taylor, Krista E. Hu, Zhenhong Hood, Mary A. Benko, Hrvoje Jonker, Tanya R. |
| contents | Gaze-based interactions offer a potential way for users to naturally engage with mixed reality (XR) interfaces. Black-box machine learning models enabled higher accuracy for gaze-based interactions. However, due to the black-box nature of the model, users might not be able to understand and effectively adapt their gaze behaviour to achieve high quality interaction. We posit that explainable AI (XAI) techniques can facilitate understanding of and interaction with gaze-based model-driven system in XR. To study this, we built a real-time, multi-level XAI interface for gaze-based interaction using a deep learning model, and evaluated it during a visual search task in XR. A between-subjects study revealed that participants who interacted with XAI made more accurate selections compared to those who did not use the XAI system (i.e., F1 score increase of 10.8%). Additionally, participants who used the XAI system adapted their gaze behavior over time to make more effective selections. These findings suggest that XAI can potentially be used to assist users in more effective collaboration with model-driven interactions in XR. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_13777 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Explainable Interfaces for Rapid Gaze-Based Interactions in Mixed Reality Yu, Mengjie Harris, Dustin Jones, Ian Zhang, Ting Liu, Yue Sendhilnathan, Naveen Kokhlikyan, Narine Wang, Fulton Tran, Co Livingston, Jordan L. Taylor, Krista E. Hu, Zhenhong Hood, Mary A. Benko, Hrvoje Jonker, Tanya R. Human-Computer Interaction Gaze-based interactions offer a potential way for users to naturally engage with mixed reality (XR) interfaces. Black-box machine learning models enabled higher accuracy for gaze-based interactions. However, due to the black-box nature of the model, users might not be able to understand and effectively adapt their gaze behaviour to achieve high quality interaction. We posit that explainable AI (XAI) techniques can facilitate understanding of and interaction with gaze-based model-driven system in XR. To study this, we built a real-time, multi-level XAI interface for gaze-based interaction using a deep learning model, and evaluated it during a visual search task in XR. A between-subjects study revealed that participants who interacted with XAI made more accurate selections compared to those who did not use the XAI system (i.e., F1 score increase of 10.8%). Additionally, participants who used the XAI system adapted their gaze behavior over time to make more effective selections. These findings suggest that XAI can potentially be used to assist users in more effective collaboration with model-driven interactions in XR. |
| title | Explainable Interfaces for Rapid Gaze-Based Interactions in Mixed Reality |
| topic | Human-Computer Interaction |
| url | https://arxiv.org/abs/2404.13777 |