Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.18672 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912601395429376 |
|---|---|
| author | Sridhar, Ajay Narayanan Qiao, Fuli Aldas, Nelson Daniel Troncoso Shi, Yanpei Mahdavi, Mehrdad Itti, Laurent Narayanan, Vijaykrishnan |
| author_facet | Sridhar, Ajay Narayanan Qiao, Fuli Aldas, Nelson Daniel Troncoso Shi, Yanpei Mahdavi, Mehrdad Itti, Laurent Narayanan, Vijaykrishnan |
| contents | People with visual impairments often face significant challenges in locating and retrieving objects in their surroundings. Existing assistive technologies present a trade-off: systems that offer precise guidance typically require pre-scanning or support only fixed object categories, while those with open-world object recognition lack spatial feedback for reaching the object. To address this gap, we introduce 'NaviSense', a mobile assistive system that combines conversational AI, vision-language models, augmented reality (AR), and LiDAR to support open-world object detection with real-time audio-haptic guidance. Users specify objects via natural language and receive continuous spatial feedback to navigate toward the target without needing prior setup. Designed with insights from a formative study and evaluated with 12 blind and low-vision participants, NaviSense significantly reduced object retrieval time and was preferred over existing tools, demonstrating the value of integrating open-world perception with precise, accessible guidance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_18672 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | NaviSense: A Multimodal Assistive Mobile application for Object Retrieval by Persons with Visual Impairment Sridhar, Ajay Narayanan Qiao, Fuli Aldas, Nelson Daniel Troncoso Shi, Yanpei Mahdavi, Mehrdad Itti, Laurent Narayanan, Vijaykrishnan Human-Computer Interaction Artificial Intelligence People with visual impairments often face significant challenges in locating and retrieving objects in their surroundings. Existing assistive technologies present a trade-off: systems that offer precise guidance typically require pre-scanning or support only fixed object categories, while those with open-world object recognition lack spatial feedback for reaching the object. To address this gap, we introduce 'NaviSense', a mobile assistive system that combines conversational AI, vision-language models, augmented reality (AR), and LiDAR to support open-world object detection with real-time audio-haptic guidance. Users specify objects via natural language and receive continuous spatial feedback to navigate toward the target without needing prior setup. Designed with insights from a formative study and evaluated with 12 blind and low-vision participants, NaviSense significantly reduced object retrieval time and was preferred over existing tools, demonstrating the value of integrating open-world perception with precise, accessible guidance. |
| title | NaviSense: A Multimodal Assistive Mobile application for Object Retrieval by Persons with Visual Impairment |
| topic | Human-Computer Interaction Artificial Intelligence |
| url | https://arxiv.org/abs/2509.18672 |