Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2511.13207 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918205703847936 |
|---|---|
| author | Peng, Cheng Zhang, Zhenzhe Chi, Cheng Wei, Xiaobao Zhang, Yanhao Wang, Heng Wang, Pengwei Wang, Zhongyuan Liu, Jing Zhang, Shanghang |
| author_facet | Peng, Cheng Zhang, Zhenzhe Chi, Cheng Wei, Xiaobao Zhang, Yanhao Wang, Heng Wang, Pengwei Wang, Zhongyuan Liu, Jing Zhang, Shanghang |
| contents | Navigating to a specified object in an unknown environment is a fundamental yet challenging capability of embodied intelligence. However, current methods struggle to balance decision frequency with intelligence, resulting in decisions lacking foresight or discontinuous actions. In this work, we propose PIGEON: Point of Interest Guided Exploration for Object Navigation with VLM, maintaining a lightweight and semantically aligned snapshot memory during exploration as semantic input for the exploration strategy. We use a large Visual-Language Model (VLM), named PIGEON-VL, to select Points of Interest (PoI) formed during exploration and then employ a lower-level planner for action output, increasing the decision frequency. Additionally, this PoI-based decision-making enables the generation of Reinforcement Learning with Verifiable Reward (RLVR) data suitable for simulators. Experiments on classic object navigation benchmarks demonstrate that our zero-shot transfer method achieves state-of-the-art performance, while RLVR further enhances the model's semantic guidance capabilities, enabling deep reasoning during real-time navigation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_13207 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | PIGEON: VLM-Driven Object Navigation via Points of Interest Selection Peng, Cheng Zhang, Zhenzhe Chi, Cheng Wei, Xiaobao Zhang, Yanhao Wang, Heng Wang, Pengwei Wang, Zhongyuan Liu, Jing Zhang, Shanghang Robotics Computer Vision and Pattern Recognition Navigating to a specified object in an unknown environment is a fundamental yet challenging capability of embodied intelligence. However, current methods struggle to balance decision frequency with intelligence, resulting in decisions lacking foresight or discontinuous actions. In this work, we propose PIGEON: Point of Interest Guided Exploration for Object Navigation with VLM, maintaining a lightweight and semantically aligned snapshot memory during exploration as semantic input for the exploration strategy. We use a large Visual-Language Model (VLM), named PIGEON-VL, to select Points of Interest (PoI) formed during exploration and then employ a lower-level planner for action output, increasing the decision frequency. Additionally, this PoI-based decision-making enables the generation of Reinforcement Learning with Verifiable Reward (RLVR) data suitable for simulators. Experiments on classic object navigation benchmarks demonstrate that our zero-shot transfer method achieves state-of-the-art performance, while RLVR further enhances the model's semantic guidance capabilities, enabling deep reasoning during real-time navigation. |
| title | PIGEON: VLM-Driven Object Navigation via Points of Interest Selection |
| topic | Robotics Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2511.13207 |