Openfly: A comprehensive platform for aerial vision-language navigation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911474834735104 |
|---|---|
| author | Gao, Yunpeng Li, Chenhui You, Zhongrui Liu, Junli Li, Zhen Chen, Pengan Chen, Qizhi Tang, Zhonghan Wang, Liansheng Yang, Penghui Tang, Yiwen Tang, Yuhang Liang, Shuai Zhu, Songyi Xiong, Ziqin Su, Yifei Ye, Xinyi Li, Jianan Ding, Yan Wang, Dong Li, Xuelong Wang, Zhigang Zhao, Bin |
| author_facet | Gao, Yunpeng Li, Chenhui You, Zhongrui Liu, Junli Li, Zhen Chen, Pengan Chen, Qizhi Tang, Zhonghan Wang, Liansheng Yang, Penghui Tang, Yiwen Tang, Yuhang Liang, Shuai Zhu, Songyi Xiong, Ziqin Su, Yifei Ye, Xinyi Li, Jianan Ding, Yan Wang, Dong Li, Xuelong Wang, Zhigang Zhao, Bin |
| contents | Vision-Language Navigation (VLN) aims to guide agents by leveraging language instructions and visual cues, playing a pivotal role in embodied AI. Indoor VLN has been extensively studied, whereas outdoor aerial VLN remains underexplored. The potential reason is that outdoor aerial view encompasses vast areas, making data collection more challenging, which results in a lack of benchmarks. To address this problem, we propose OpenFly, a platform comprising various rendering engines, a versatile toolchain, and a large-scale benchmark for aerial VLN. Firstly, we integrate diverse rendering engines and advanced techniques for environment simulation, including Unreal Engine, GTA V, Google Earth, and 3D Gaussian Splatting (3D GS). Particularly, 3D GS supports real-to-sim rendering, further enhancing the realism of our environments. Secondly, we develop a highly automated toolchain for aerial VLN data collection, streamlining point cloud acquisition, scene semantic segmentation, flight trajectory creation, and instruction generation. Thirdly, based on the toolchain, we construct a large-scale aerial VLN dataset with 100k trajectories, covering diverse heights and lengths across 18 scenes. Moreover, we propose OpenFly-Agent, a keyframe-aware VLN model emphasizing key observations during flight. For benchmarking, extensive experiments and analyses are conducted, evaluating several recent VLN methods and showcasing the superiority of our OpenFly platform and agent. The toolchain, dataset, and codes will be open-sourced. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_18041 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Openfly: A comprehensive platform for aerial vision-language navigation Gao, Yunpeng Li, Chenhui You, Zhongrui Liu, Junli Li, Zhen Chen, Pengan Chen, Qizhi Tang, Zhonghan Wang, Liansheng Yang, Penghui Tang, Yiwen Tang, Yuhang Liang, Shuai Zhu, Songyi Xiong, Ziqin Su, Yifei Ye, Xinyi Li, Jianan Ding, Yan Wang, Dong Li, Xuelong Wang, Zhigang Zhao, Bin Computer Vision and Pattern Recognition Robotics Vision-Language Navigation (VLN) aims to guide agents by leveraging language instructions and visual cues, playing a pivotal role in embodied AI. Indoor VLN has been extensively studied, whereas outdoor aerial VLN remains underexplored. The potential reason is that outdoor aerial view encompasses vast areas, making data collection more challenging, which results in a lack of benchmarks. To address this problem, we propose OpenFly, a platform comprising various rendering engines, a versatile toolchain, and a large-scale benchmark for aerial VLN. Firstly, we integrate diverse rendering engines and advanced techniques for environment simulation, including Unreal Engine, GTA V, Google Earth, and 3D Gaussian Splatting (3D GS). Particularly, 3D GS supports real-to-sim rendering, further enhancing the realism of our environments. Secondly, we develop a highly automated toolchain for aerial VLN data collection, streamlining point cloud acquisition, scene semantic segmentation, flight trajectory creation, and instruction generation. Thirdly, based on the toolchain, we construct a large-scale aerial VLN dataset with 100k trajectories, covering diverse heights and lengths across 18 scenes. Moreover, we propose OpenFly-Agent, a keyframe-aware VLN model emphasizing key observations during flight. For benchmarking, extensive experiments and analyses are conducted, evaluating several recent VLN methods and showcasing the superiority of our OpenFly platform and agent. The toolchain, dataset, and codes will be open-sourced. |
| title | Openfly: A comprehensive platform for aerial vision-language navigation |
| topic | Computer Vision and Pattern Recognition Robotics |
| url | https://arxiv.org/abs/2502.18041 |