Towards Deeper Understanding of Natural User Interactions in Virtual Reality Based Assembly Tasks
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915460242472960 |
|---|---|
| author | Ghamandi, Ryan Hmaiti, Yahya Maslych, Mykola Kattoju, Ravi Kiran LaViola Jr, Joseph J. |
| author_facet | Ghamandi, Ryan Hmaiti, Yahya Maslych, Mykola Kattoju, Ravi Kiran LaViola Jr, Joseph J. |
| contents | We explore natural user interactions using a virtual reality simulation of a robot arm for assembly tasks. Using a Wizard-of-Oz study, participants completed collaborative LEGO and instructive PCB assembly tasks, with the robot responding under experimenter control. We collected voice, hand tracking, and gaze data from users. Statistical analyses revealed that instructive and collaborative scenarios elicit distinct behaviors and adopted strategies, particularly as tasks progress. Users tended to use put-that-there language in spatially ambiguous contexts and more descriptive instructions in spatially clear ones. Our contributions include the identification of natural interaction strategies through analyses of collected data, as well as the supporting dataset, to guide the understanding and design of natural multimodal user interfaces for instructive interaction with systems in virtual reality. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_17124 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards Deeper Understanding of Natural User Interactions in Virtual Reality Based Assembly Tasks Ghamandi, Ryan Hmaiti, Yahya Maslych, Mykola Kattoju, Ravi Kiran LaViola Jr, Joseph J. Human-Computer Interaction Systems and Control We explore natural user interactions using a virtual reality simulation of a robot arm for assembly tasks. Using a Wizard-of-Oz study, participants completed collaborative LEGO and instructive PCB assembly tasks, with the robot responding under experimenter control. We collected voice, hand tracking, and gaze data from users. Statistical analyses revealed that instructive and collaborative scenarios elicit distinct behaviors and adopted strategies, particularly as tasks progress. Users tended to use put-that-there language in spatially ambiguous contexts and more descriptive instructions in spatially clear ones. Our contributions include the identification of natural interaction strategies through analyses of collected data, as well as the supporting dataset, to guide the understanding and design of natural multimodal user interfaces for instructive interaction with systems in virtual reality. |
| title | Towards Deeper Understanding of Natural User Interactions in Virtual Reality Based Assembly Tasks |
| topic | Human-Computer Interaction Systems and Control |
| url | https://arxiv.org/abs/2508.17124 |