Coordinated 2D-3D Visualization of Volumetric Medical Data in XR with Multimodal Interactions
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915363983196160 |
|---|---|
| author | Liu, Qixuan Qiu, Shi Wang, Yinqiao Wu, Xiwen Chok, Kenneth Siu Ho Fu, Chi-Wing Heng, Pheng-Ann |
| author_facet | Liu, Qixuan Qiu, Shi Wang, Yinqiao Wu, Xiwen Chok, Kenneth Siu Ho Fu, Chi-Wing Heng, Pheng-Ann |
| contents | Volumetric medical imaging technologies produce detailed 3D representations of anatomical structures. However, effective medical data visualization and exploration pose significant challenges, especially for individuals with limited medical expertise. We introduce a novel XR-based system with two key innovations: (1) a coordinated visualization module integrating Multi-layered Multi-planar Reconstruction with 3D mesh models and (2) a multimodal interaction framework combining hand gestures with LLM-enabled voice commands. We conduct preliminary evaluations, including a 15-participant user study and expert interviews, to demonstrate the system's abilities to enhance spatial understanding and reduce cognitive load. Experimental results show notable improvements in task completion times, usability metrics, and interaction effectiveness enhanced by LLM-driven voice control. While identifying areas for future refinement, our findings highlight the potential of this immersive visualization system to advance medical training and clinical practice. Our demo application and supplemental materials are available for download at: https://osf.io/bpjq5/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_22926 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Coordinated 2D-3D Visualization of Volumetric Medical Data in XR with Multimodal Interactions Liu, Qixuan Qiu, Shi Wang, Yinqiao Wu, Xiwen Chok, Kenneth Siu Ho Fu, Chi-Wing Heng, Pheng-Ann Human-Computer Interaction Graphics Multimedia Volumetric medical imaging technologies produce detailed 3D representations of anatomical structures. However, effective medical data visualization and exploration pose significant challenges, especially for individuals with limited medical expertise. We introduce a novel XR-based system with two key innovations: (1) a coordinated visualization module integrating Multi-layered Multi-planar Reconstruction with 3D mesh models and (2) a multimodal interaction framework combining hand gestures with LLM-enabled voice commands. We conduct preliminary evaluations, including a 15-participant user study and expert interviews, to demonstrate the system's abilities to enhance spatial understanding and reduce cognitive load. Experimental results show notable improvements in task completion times, usability metrics, and interaction effectiveness enhanced by LLM-driven voice control. While identifying areas for future refinement, our findings highlight the potential of this immersive visualization system to advance medical training and clinical practice. Our demo application and supplemental materials are available for download at: https://osf.io/bpjq5/. |
| title | Coordinated 2D-3D Visualization of Volumetric Medical Data in XR with Multimodal Interactions |
| topic | Human-Computer Interaction Graphics Multimedia |
| url | https://arxiv.org/abs/2506.22926 |