Eyes on the Game: Deciphering Implicit Human Signals to Infer Human Proficiency, Trust, and Intent
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910511920054272 |
|---|---|
| author | Hulle, Nikhil Aroca-Ouellette, Stéphane Ries, Anthony J. Brawer, Jake von der Wense, Katharina Roncone, Alessandro |
| author_facet | Hulle, Nikhil Aroca-Ouellette, Stéphane Ries, Anthony J. Brawer, Jake von der Wense, Katharina Roncone, Alessandro |
| contents | Effective collaboration between humans and AIs hinges on transparent communication and alignment of mental models. However, explicit, verbal communication is not always feasible. Under such circumstances, human-human teams often depend on implicit, nonverbal cues to glean important information about their teammates such as intent and expertise, thereby bolstering team alignment and adaptability. Among these implicit cues, two of the most salient and fundamental are a human's actions in the environment and their visual attention. In this paper, we present a novel method to combine eye gaze data and behavioral data, and evaluate their respective predictive power for human proficiency, trust, and intent. We first collect a dataset of paired eye gaze and gameplay data in the fast-paced collaborative "Overcooked" environment. We then train models on this dataset to compare how the predictive powers differ between gaze data, gameplay data, and their combination. We additionally compare our method to prior works that aggregate eye gaze data and demonstrate how these aggregation methods can substantially reduce the predictive ability of eye gaze. Our results indicate that, while eye gaze data and gameplay data excel in different situations, a model that integrates both types consistently outperforms all baselines. This work paves the way for developing intuitive and responsive agents that can efficiently adapt to new teammates. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_03298 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Eyes on the Game: Deciphering Implicit Human Signals to Infer Human Proficiency, Trust, and Intent Hulle, Nikhil Aroca-Ouellette, Stéphane Ries, Anthony J. Brawer, Jake von der Wense, Katharina Roncone, Alessandro Human-Computer Interaction Effective collaboration between humans and AIs hinges on transparent communication and alignment of mental models. However, explicit, verbal communication is not always feasible. Under such circumstances, human-human teams often depend on implicit, nonverbal cues to glean important information about their teammates such as intent and expertise, thereby bolstering team alignment and adaptability. Among these implicit cues, two of the most salient and fundamental are a human's actions in the environment and their visual attention. In this paper, we present a novel method to combine eye gaze data and behavioral data, and evaluate their respective predictive power for human proficiency, trust, and intent. We first collect a dataset of paired eye gaze and gameplay data in the fast-paced collaborative "Overcooked" environment. We then train models on this dataset to compare how the predictive powers differ between gaze data, gameplay data, and their combination. We additionally compare our method to prior works that aggregate eye gaze data and demonstrate how these aggregation methods can substantially reduce the predictive ability of eye gaze. Our results indicate that, while eye gaze data and gameplay data excel in different situations, a model that integrates both types consistently outperforms all baselines. This work paves the way for developing intuitive and responsive agents that can efficiently adapt to new teammates. |
| title | Eyes on the Game: Deciphering Implicit Human Signals to Infer Human Proficiency, Trust, and Intent |
| topic | Human-Computer Interaction |
| url | https://arxiv.org/abs/2407.03298 |