Beyond the Monitor: Mixed Reality Visualization and Multimodal AI for Enhanced Digital Pathology Workflow
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866918356938915840 |
|---|---|
| author | Veerla, Jai Prakash Guttikonda, Partha Sai Shang, Helen H. Nasr, Mohammad Sadegh Torres, Cesar Luber, Jacob M. |
| author_facet | Veerla, Jai Prakash Guttikonda, Partha Sai Shang, Helen H. Nasr, Mohammad Sadegh Torres, Cesar Luber, Jacob M. |
| contents | Pathologists diagnose cancer using gigapixel whole-slide images (WSIs), but the current digital workflow is fragmented. These multiscale datasets often exceed 100,000 x 100,000 pixels, yet standard 2D monitors restrict the field of view. This disparity forces constant panning and zooming, which increases cognitive load and disrupts diagnostic momentum. We introduce PathVis, a mixed-reality platform for Apple Vision Pro that unifies this ecosystem into a single immersive environment. PathVis replaces indirect mouse navigation with embodied interaction, utilizing eye gaze, natural hand gestures, and voice commands to explore gigapixel data. The system integrates multimodal AI agents to support computer-aided diagnosis: a content-based image retrieval engine spatially displays similar patient cases for side-by-side prognostic comparison, while a conversational assistant provides real-time interpretation. By merging immersive visualization with integrated AI capabilities, PathVis shows promise in streamlining diagnostic workflows and mitigating the burden of context switching. This paper presents the system architecture and a preliminary qualitative evaluation demonstrating the platform's feasibility. The PathVis source code and a demo video are publicly available at: https://github.com/jaiprakash1824/Path_Vis. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_02780 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Beyond the Monitor: Mixed Reality Visualization and Multimodal AI for Enhanced Digital Pathology Workflow Veerla, Jai Prakash Guttikonda, Partha Sai Shang, Helen H. Nasr, Mohammad Sadegh Torres, Cesar Luber, Jacob M. Human-Computer Interaction Artificial Intelligence Emerging Technologies Tissues and Organs Pathologists diagnose cancer using gigapixel whole-slide images (WSIs), but the current digital workflow is fragmented. These multiscale datasets often exceed 100,000 x 100,000 pixels, yet standard 2D monitors restrict the field of view. This disparity forces constant panning and zooming, which increases cognitive load and disrupts diagnostic momentum. We introduce PathVis, a mixed-reality platform for Apple Vision Pro that unifies this ecosystem into a single immersive environment. PathVis replaces indirect mouse navigation with embodied interaction, utilizing eye gaze, natural hand gestures, and voice commands to explore gigapixel data. The system integrates multimodal AI agents to support computer-aided diagnosis: a content-based image retrieval engine spatially displays similar patient cases for side-by-side prognostic comparison, while a conversational assistant provides real-time interpretation. By merging immersive visualization with integrated AI capabilities, PathVis shows promise in streamlining diagnostic workflows and mitigating the burden of context switching. This paper presents the system architecture and a preliminary qualitative evaluation demonstrating the platform's feasibility. The PathVis source code and a demo video are publicly available at: https://github.com/jaiprakash1824/Path_Vis. |
| title | Beyond the Monitor: Mixed Reality Visualization and Multimodal AI for Enhanced Digital Pathology Workflow |
| topic | Human-Computer Interaction Artificial Intelligence Emerging Technologies Tissues and Organs |
| url | https://arxiv.org/abs/2505.02780 |