Adaptive 3D UI Placement in Mixed Reality Using Deep Reinforcement Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908344031117312 |
|---|---|
| author | Lu, Feiyu Chen, Mengyu Hsu, Hsiang Deshpande, Pranav Wang, Cheng Yao MacIntyre, Blair |
| author_facet | Lu, Feiyu Chen, Mengyu Hsu, Hsiang Deshpande, Pranav Wang, Cheng Yao MacIntyre, Blair |
| contents | Mixed Reality (MR) could assist users' tasks by continuously integrating virtual content with their view of the physical environment. However, where and how to place these content to best support the users has been a challenging problem due to the dynamic nature of MR experiences. In contrast to prior work that investigates optimization-based methods, we are exploring how reinforcement learning (RL) could assist with continuous 3D content placement that is aware of users' poses and their surrounding environments. Through an initial exploration and preliminary evaluation, our results demonstrate the potential of RL to position content that maximizes the reward for users on the go. We further identify future directions for research that could harness the power of RL for personalized and optimized UI and content placement in MR. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_21731 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Adaptive 3D UI Placement in Mixed Reality Using Deep Reinforcement Learning Lu, Feiyu Chen, Mengyu Hsu, Hsiang Deshpande, Pranav Wang, Cheng Yao MacIntyre, Blair Human-Computer Interaction Artificial Intelligence Computer Vision and Pattern Recognition Mixed Reality (MR) could assist users' tasks by continuously integrating virtual content with their view of the physical environment. However, where and how to place these content to best support the users has been a challenging problem due to the dynamic nature of MR experiences. In contrast to prior work that investigates optimization-based methods, we are exploring how reinforcement learning (RL) could assist with continuous 3D content placement that is aware of users' poses and their surrounding environments. Through an initial exploration and preliminary evaluation, our results demonstrate the potential of RL to position content that maximizes the reward for users on the go. We further identify future directions for research that could harness the power of RL for personalized and optimized UI and content placement in MR. |
| title | Adaptive 3D UI Placement in Mixed Reality Using Deep Reinforcement Learning |
| topic | Human-Computer Interaction Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2504.21731 |