Eye Gaze as a Signal for Conveying User Attention in Contextual AI Systems
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908316306767872 |
|---|---|
| author | Wilson, Ethan Sendhilnathan, Naveen Burlingham, Charlie S. Mansour, Yusuf Cavin, Robert Tetali, Sai Deep Fernandes, Ajoy Savio Proulx, Michael J. |
| author_facet | Wilson, Ethan Sendhilnathan, Naveen Burlingham, Charlie S. Mansour, Yusuf Cavin, Robert Tetali, Sai Deep Fernandes, Ajoy Savio Proulx, Michael J. |
| contents | Advanced multimodal AI agents can now collaborate with users to solve challenges in the world. Yet, these emerging contextual AI systems rely on explicit communication channels between the user and system. We hypothesize that implicit communication of the user's interests and intent would reduce friction and improve user experience when collaborating with AI agents. In this work, we explore the potential of wearable eye tracking to convey signals about user attention. We measure the eye tracking signal quality requirements to effectively map gaze traces to physical objects, then conduct experiments that provide visual scanpath history as additional context when querying vision language models. Our results show that eye tracking provides high value as a user attention signal and can convey important context about the user's current task and interests, improving understanding of contextual AI agents. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_13878 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Eye Gaze as a Signal for Conveying User Attention in Contextual AI Systems Wilson, Ethan Sendhilnathan, Naveen Burlingham, Charlie S. Mansour, Yusuf Cavin, Robert Tetali, Sai Deep Fernandes, Ajoy Savio Proulx, Michael J. Human-Computer Interaction Computer Vision and Pattern Recognition Advanced multimodal AI agents can now collaborate with users to solve challenges in the world. Yet, these emerging contextual AI systems rely on explicit communication channels between the user and system. We hypothesize that implicit communication of the user's interests and intent would reduce friction and improve user experience when collaborating with AI agents. In this work, we explore the potential of wearable eye tracking to convey signals about user attention. We measure the eye tracking signal quality requirements to effectively map gaze traces to physical objects, then conduct experiments that provide visual scanpath history as additional context when querying vision language models. Our results show that eye tracking provides high value as a user attention signal and can convey important context about the user's current task and interests, improving understanding of contextual AI agents. |
| title | Eye Gaze as a Signal for Conveying User Attention in Contextual AI Systems |
| topic | Human-Computer Interaction Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2501.13878 |