Fine-Tuning Vision-Language Models for Visual Navigation Assistance
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918138140950528 |
|---|---|
| author | Li, Xiao Gandhi, Bharat Zhan, Ming Nehra, Mohit Zhang, Zhicheng Sun, Yuchen Song, Meijia Zhang, Naisheng Wang, Xi |
| author_facet | Li, Xiao Gandhi, Bharat Zhan, Ming Nehra, Mohit Zhang, Zhicheng Sun, Yuchen Song, Meijia Zhang, Naisheng Wang, Xi |
| contents | We address vision-language-driven indoor navigation to assist visually impaired individuals in reaching a target location using images and natural language guidance. Traditional navigation systems are ineffective indoors due to the lack of precise location data. Our approach integrates vision and language models to generate step-by-step navigational instructions, enhancing accessibility and independence. We fine-tune the BLIP-2 model with Low Rank Adaptation (LoRA) on a manually annotated indoor navigation dataset. We propose an evaluation metric that refines the BERT F1 score by emphasizing directional and sequential variables, providing a more comprehensive measure of navigational performance. After applying LoRA, the model significantly improved in generating directional instructions, overcoming limitations in the original BLIP-2 model. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_07488 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Fine-Tuning Vision-Language Models for Visual Navigation Assistance Li, Xiao Gandhi, Bharat Zhan, Ming Nehra, Mohit Zhang, Zhicheng Sun, Yuchen Song, Meijia Zhang, Naisheng Wang, Xi Computer Vision and Pattern Recognition Artificial Intelligence We address vision-language-driven indoor navigation to assist visually impaired individuals in reaching a target location using images and natural language guidance. Traditional navigation systems are ineffective indoors due to the lack of precise location data. Our approach integrates vision and language models to generate step-by-step navigational instructions, enhancing accessibility and independence. We fine-tune the BLIP-2 model with Low Rank Adaptation (LoRA) on a manually annotated indoor navigation dataset. We propose an evaluation metric that refines the BERT F1 score by emphasizing directional and sequential variables, providing a more comprehensive measure of navigational performance. After applying LoRA, the model significantly improved in generating directional instructions, overcoming limitations in the original BLIP-2 model. |
| title | Fine-Tuning Vision-Language Models for Visual Navigation Assistance |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2509.07488 |