Can You Move These Over There? An LLM-based VR Mover for Supporting Object Manipulation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915137111195648 |
|---|---|
| author | Wang, Xiangzhi Eric Sin, Zackary P. T. Jia, Ye Archer, Daniel Fong, Wynonna H. Y. Li, Qing Li, Chen |
| author_facet | Wang, Xiangzhi Eric Sin, Zackary P. T. Jia, Ye Archer, Daniel Fong, Wynonna H. Y. Li, Qing Li, Chen |
| contents | In our daily lives, we can naturally convey instructions for the spatial manipulation of objects using words and gestures. Transposing this form of interaction into virtual reality (VR) object manipulation can be beneficial. We propose VR Mover, an LLM-empowered solution that can understand and interpret the user's vocal instruction to support object manipulation. By simply pointing and speaking, the LLM can manipulate objects without structured input. Our user study demonstrates that VR Mover enhances user usability, overall experience and performance on multi-object manipulation, while also reducing workload and arm fatigue. Users prefer the proposed natural interface for broad movements and may complementarily switch to gizmos or virtual hands for finer adjustments. These findings are believed to contribute to design implications for future LLM-based object manipulation interfaces, highlighting the potential for more intuitive and efficient user interactions in VR environments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_02201 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Can You Move These Over There? An LLM-based VR Mover for Supporting Object Manipulation Wang, Xiangzhi Eric Sin, Zackary P. T. Jia, Ye Archer, Daniel Fong, Wynonna H. Y. Li, Qing Li, Chen Human-Computer Interaction Artificial Intelligence Computation and Language Emerging Technologies In our daily lives, we can naturally convey instructions for the spatial manipulation of objects using words and gestures. Transposing this form of interaction into virtual reality (VR) object manipulation can be beneficial. We propose VR Mover, an LLM-empowered solution that can understand and interpret the user's vocal instruction to support object manipulation. By simply pointing and speaking, the LLM can manipulate objects without structured input. Our user study demonstrates that VR Mover enhances user usability, overall experience and performance on multi-object manipulation, while also reducing workload and arm fatigue. Users prefer the proposed natural interface for broad movements and may complementarily switch to gizmos or virtual hands for finer adjustments. These findings are believed to contribute to design implications for future LLM-based object manipulation interfaces, highlighting the potential for more intuitive and efficient user interactions in VR environments. |
| title | Can You Move These Over There? An LLM-based VR Mover for Supporting Object Manipulation |
| topic | Human-Computer Interaction Artificial Intelligence Computation and Language Emerging Technologies |
| url | https://arxiv.org/abs/2502.02201 |