Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.13250 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911319576281088 |
|---|---|
| author | Koo, Juil Choi, Daehyeon Youn, Sangwoo Lee, Phillip Y. Sung, Minhyuk |
| author_facet | Koo, Juil Choi, Daehyeon Youn, Sangwoo Lee, Phillip Y. Sung, Minhyuk |
| contents | Vision Language Models (VLMs) excel at visual question answering (VQA) but remain limited to snapshot vision, reasoning from static images. In contrast, embodied agents require ambulatory vision, actively moving to obtain more informative views. We introduce Visually Grounded Active View Selection (VG-AVS), a task that selects the most informative next viewpoint using only the visual information in the current image, without relying on scene memory or external knowledge. To support this task, we construct a synthetic dataset with automatically generated paired query-target views and question-answer prompts. We also propose a framework that fine-tunes pretrained VLMs through supervised fine-tuning (SFT) followed by RL-based policy optimization. Our approach achieves strong question answering performance based on viewpoint selection and generalizes robustly to unseen synthetic and real scenes. Furthermore, incorporating our learned VG-AVS framework into existing scene-exploration-based EQA systems improves downstream question-answering accuracy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_13250 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Toward Ambulatory Vision: Learning Visually-Grounded Active View Selection Koo, Juil Choi, Daehyeon Youn, Sangwoo Lee, Phillip Y. Sung, Minhyuk Computer Vision and Pattern Recognition Vision Language Models (VLMs) excel at visual question answering (VQA) but remain limited to snapshot vision, reasoning from static images. In contrast, embodied agents require ambulatory vision, actively moving to obtain more informative views. We introduce Visually Grounded Active View Selection (VG-AVS), a task that selects the most informative next viewpoint using only the visual information in the current image, without relying on scene memory or external knowledge. To support this task, we construct a synthetic dataset with automatically generated paired query-target views and question-answer prompts. We also propose a framework that fine-tunes pretrained VLMs through supervised fine-tuning (SFT) followed by RL-based policy optimization. Our approach achieves strong question answering performance based on viewpoint selection and generalizes robustly to unseen synthetic and real scenes. Furthermore, incorporating our learned VG-AVS framework into existing scene-exploration-based EQA systems improves downstream question-answering accuracy. |
| title | Toward Ambulatory Vision: Learning Visually-Grounded Active View Selection |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2512.13250 |