Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2507.18374 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911074840739840 |
|---|---|
| author | Bellos, Filippos Li, Yayuan Shu, Cary Day, Ruey Siskind, Jeffrey M. Corso, Jason J. |
| author_facet | Bellos, Filippos Li, Yayuan Shu, Cary Day, Ruey Siskind, Jeffrey M. Corso, Jason J. |
| contents | Effective human-AI collaboration for physical task completion has significant potential in both everyday activities and professional domains. AI agents equipped with informative guidance can enhance human performance, but evaluating such collaboration remains challenging due to the complexity of human-in-the-loop interactions. In this work, we introduce an evaluation framework and a multimodal dataset of human-AI interactions designed to assess how AI guidance affects procedural task performance, error reduction and learning outcomes. Besides, we develop an augmented reality (AR)-equipped AI agent that provides interactive guidance in real-world tasks, from cooking to battlefield medicine. Through human studies, we share empirical insights into AI-assisted human performance and demonstrate that AI-assisted collaboration improves task completion. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_18374 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards Effective Human-in-the-Loop Assistive AI Agents Bellos, Filippos Li, Yayuan Shu, Cary Day, Ruey Siskind, Jeffrey M. Corso, Jason J. Computer Vision and Pattern Recognition Effective human-AI collaboration for physical task completion has significant potential in both everyday activities and professional domains. AI agents equipped with informative guidance can enhance human performance, but evaluating such collaboration remains challenging due to the complexity of human-in-the-loop interactions. In this work, we introduce an evaluation framework and a multimodal dataset of human-AI interactions designed to assess how AI guidance affects procedural task performance, error reduction and learning outcomes. Besides, we develop an augmented reality (AR)-equipped AI agent that provides interactive guidance in real-world tasks, from cooking to battlefield medicine. Through human studies, we share empirical insights into AI-assisted human performance and demonstrate that AI-assisted collaboration improves task completion. |
| title | Towards Effective Human-in-the-Loop Assistive AI Agents |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2507.18374 |