Satori: Towards Proactive AR Assistant with Belief-Desire-Intention User Modeling
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916667278229504 |
|---|---|
| author | Li, Chenyi Wu, Guande Chan, Gromit Yeuk-Yin Turakhia, Dishita G Quispe, Sonia Castelo Li, Dong Welch, Leslie Silva, Claudio Qian, Jing |
| author_facet | Li, Chenyi Wu, Guande Chan, Gromit Yeuk-Yin Turakhia, Dishita G Quispe, Sonia Castelo Li, Dong Welch, Leslie Silva, Claudio Qian, Jing |
| contents | Augmented Reality (AR) assistance is increasingly used for supporting users with physical tasks like assembly and cooking. However, most systems rely on reactive responses triggered by user input, overlooking rich contextual and user-specific information. To address this, we present Satori, a novel AR system that proactively guides users by modeling both -- their mental states and environmental contexts. Satori integrates the Belief-Desire-Intention (BDI) framework with the state-of-the-art multi-modal large language model (LLM) to deliver contextually appropriate guidance. Our system is designed based on two formative studies involving twelve experts. We evaluated the system with a sixteen within-subject study and found that Satori matches the performance of designer-created Wizard-of-Oz (WoZ) systems, without manual configurations or heuristics, thereby improving generalizability, reusability, and expanding the potential of AR assistance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_16668 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Satori: Towards Proactive AR Assistant with Belief-Desire-Intention User Modeling Li, Chenyi Wu, Guande Chan, Gromit Yeuk-Yin Turakhia, Dishita G Quispe, Sonia Castelo Li, Dong Welch, Leslie Silva, Claudio Qian, Jing Human-Computer Interaction Artificial Intelligence Augmented Reality (AR) assistance is increasingly used for supporting users with physical tasks like assembly and cooking. However, most systems rely on reactive responses triggered by user input, overlooking rich contextual and user-specific information. To address this, we present Satori, a novel AR system that proactively guides users by modeling both -- their mental states and environmental contexts. Satori integrates the Belief-Desire-Intention (BDI) framework with the state-of-the-art multi-modal large language model (LLM) to deliver contextually appropriate guidance. Our system is designed based on two formative studies involving twelve experts. We evaluated the system with a sixteen within-subject study and found that Satori matches the performance of designer-created Wizard-of-Oz (WoZ) systems, without manual configurations or heuristics, thereby improving generalizability, reusability, and expanding the potential of AR assistance. |
| title | Satori: Towards Proactive AR Assistant with Belief-Desire-Intention User Modeling |
| topic | Human-Computer Interaction Artificial Intelligence |
| url | https://arxiv.org/abs/2410.16668 |