Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.22653 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915554474852352 |
|---|---|
| author | Hu, Chih Yao Lin, Yang-Sen Lee, Yuna Su, Chih-Hai Lee, Jie-Ying Tsai, Shr-Ruei Lin, Chin-Yang Chen, Kuan-Wen Ke, Tsung-Wei Liu, Yu-Lun |
| author_facet | Hu, Chih Yao Lin, Yang-Sen Lee, Yuna Su, Chih-Hai Lee, Jie-Ying Tsai, Shr-Ruei Lin, Chin-Yang Chen, Kuan-Wen Ke, Tsung-Wei Liu, Yu-Lun |
| contents | We present See, Point, Fly (SPF), a training-free aerial vision-and-language navigation (AVLN) framework built atop vision-language models (VLMs). SPF is capable of navigating to any goal based on any type of free-form instructions in any kind of environment. In contrast to existing VLM-based approaches that treat action prediction as a text generation task, our key insight is to consider action prediction for AVLN as a 2D spatial grounding task. SPF harnesses VLMs to decompose vague language instructions into iterative annotation of 2D waypoints on the input image. Along with the predicted traveling distance, SPF transforms predicted 2D waypoints into 3D displacement vectors as action commands for UAVs. Moreover, SPF also adaptively adjusts the traveling distance to facilitate more efficient navigation. Notably, SPF performs navigation in a closed-loop control manner, enabling UAVs to follow dynamic targets in dynamic environments. SPF sets a new state of the art in DRL simulation benchmark, outperforming the previous best method by an absolute margin of 63%. In extensive real-world evaluations, SPF outperforms strong baselines by a large margin. We also conduct comprehensive ablation studies to highlight the effectiveness of our design choice. Lastly, SPF shows remarkable generalization to different VLMs. Project page: https://spf-web.pages.dev |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_22653 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | See, Point, Fly: A Learning-Free VLM Framework for Universal Unmanned Aerial Navigation Hu, Chih Yao Lin, Yang-Sen Lee, Yuna Su, Chih-Hai Lee, Jie-Ying Tsai, Shr-Ruei Lin, Chin-Yang Chen, Kuan-Wen Ke, Tsung-Wei Liu, Yu-Lun Robotics Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition Machine Learning We present See, Point, Fly (SPF), a training-free aerial vision-and-language navigation (AVLN) framework built atop vision-language models (VLMs). SPF is capable of navigating to any goal based on any type of free-form instructions in any kind of environment. In contrast to existing VLM-based approaches that treat action prediction as a text generation task, our key insight is to consider action prediction for AVLN as a 2D spatial grounding task. SPF harnesses VLMs to decompose vague language instructions into iterative annotation of 2D waypoints on the input image. Along with the predicted traveling distance, SPF transforms predicted 2D waypoints into 3D displacement vectors as action commands for UAVs. Moreover, SPF also adaptively adjusts the traveling distance to facilitate more efficient navigation. Notably, SPF performs navigation in a closed-loop control manner, enabling UAVs to follow dynamic targets in dynamic environments. SPF sets a new state of the art in DRL simulation benchmark, outperforming the previous best method by an absolute margin of 63%. In extensive real-world evaluations, SPF outperforms strong baselines by a large margin. We also conduct comprehensive ablation studies to highlight the effectiveness of our design choice. Lastly, SPF shows remarkable generalization to different VLMs. Project page: https://spf-web.pages.dev |
| title | See, Point, Fly: A Learning-Free VLM Framework for Universal Unmanned Aerial Navigation |
| topic | Robotics Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2509.22653 |