MR.NAVI: Mixed-Reality Navigation Assistant for the Visually Impaired
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908395355766784 |
|---|---|
| author | Pfitzer, Nicolas Zhou, Yifan Poggensee, Marco Kurtulus, Defne Dominguez-Dager, Bessie Dusmanu, Mihai Pollefeys, Marc Bauer, Zuria |
| author_facet | Pfitzer, Nicolas Zhou, Yifan Poggensee, Marco Kurtulus, Defne Dominguez-Dager, Bessie Dusmanu, Mihai Pollefeys, Marc Bauer, Zuria |
| contents | Over 43 million people worldwide live with severe visual impairment, facing significant challenges in navigating unfamiliar environments. We present MR.NAVI, a mixed reality system that enhances spatial awareness for visually impaired users through real-time scene understanding and intuitive audio feedback. Our system combines computer vision algorithms for object detection and depth estimation with natural language processing to provide contextual scene descriptions, proactive collision avoidance, and navigation instructions. The distributed architecture processes sensor data through MobileNet for object detection and employs RANSAC-based floor detection with DBSCAN clustering for obstacle avoidance. Integration with public transit APIs enables navigation with public transportation directions. Through our experiments with user studies, we evaluated both scene description and navigation functionalities in unfamiliar environments, showing promising usability and effectiveness. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_05369 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MR.NAVI: Mixed-Reality Navigation Assistant for the Visually Impaired Pfitzer, Nicolas Zhou, Yifan Poggensee, Marco Kurtulus, Defne Dominguez-Dager, Bessie Dusmanu, Mihai Pollefeys, Marc Bauer, Zuria Computer Vision and Pattern Recognition Over 43 million people worldwide live with severe visual impairment, facing significant challenges in navigating unfamiliar environments. We present MR.NAVI, a mixed reality system that enhances spatial awareness for visually impaired users through real-time scene understanding and intuitive audio feedback. Our system combines computer vision algorithms for object detection and depth estimation with natural language processing to provide contextual scene descriptions, proactive collision avoidance, and navigation instructions. The distributed architecture processes sensor data through MobileNet for object detection and employs RANSAC-based floor detection with DBSCAN clustering for obstacle avoidance. Integration with public transit APIs enables navigation with public transportation directions. Through our experiments with user studies, we evaluated both scene description and navigation functionalities in unfamiliar environments, showing promising usability and effectiveness. |
| title | MR.NAVI: Mixed-Reality Navigation Assistant for the Visually Impaired |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2506.05369 |