NeRF-VINS: A Real-time Neural Radiance Field Map-based Visual-Inertial Navigation System
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910358081372160 |
|---|---|
| author | Katragadda, Saimouli Lee, Woosik Peng, Yuxiang Geneva, Patrick Chen, Chuchu Guo, Chao Li, Mingyang Huang, Guoquan |
| author_facet | Katragadda, Saimouli Lee, Woosik Peng, Yuxiang Geneva, Patrick Chen, Chuchu Guo, Chao Li, Mingyang Huang, Guoquan |
| contents | Achieving efficient and consistent localization a prior map remains challenging in robotics. Conventional keyframe-based approaches often suffers from sub-optimal viewpoints due to limited field of view (FOV) and/or constrained motion, thus degrading the localization performance. To address this issue, we design a real-time tightly-coupled Neural Radiance Fields (NeRF)-aided visual-inertial navigation system (VINS). In particular, by effectively leveraging the NeRF's potential to synthesize novel views, the proposed NeRF-VINS overcomes the limitations of traditional keyframe-based maps (with limited views) and optimally fuses IMU, monocular images, and synthetically rendered images within an efficient filter-based framework. This tightly-coupled fusion enables efficient 3D motion tracking with bounded errors. We extensively compare the proposed NeRF-VINS against the state-of-the-art methods that use prior map information and demonstrate its ability to perform real-time localization, at over 10 Hz, on a resource-constrained Jetson AGX Orin embedded platform. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_09295 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | NeRF-VINS: A Real-time Neural Radiance Field Map-based Visual-Inertial Navigation System Katragadda, Saimouli Lee, Woosik Peng, Yuxiang Geneva, Patrick Chen, Chuchu Guo, Chao Li, Mingyang Huang, Guoquan Robotics Achieving efficient and consistent localization a prior map remains challenging in robotics. Conventional keyframe-based approaches often suffers from sub-optimal viewpoints due to limited field of view (FOV) and/or constrained motion, thus degrading the localization performance. To address this issue, we design a real-time tightly-coupled Neural Radiance Fields (NeRF)-aided visual-inertial navigation system (VINS). In particular, by effectively leveraging the NeRF's potential to synthesize novel views, the proposed NeRF-VINS overcomes the limitations of traditional keyframe-based maps (with limited views) and optimally fuses IMU, monocular images, and synthetically rendered images within an efficient filter-based framework. This tightly-coupled fusion enables efficient 3D motion tracking with bounded errors. We extensively compare the proposed NeRF-VINS against the state-of-the-art methods that use prior map information and demonstrate its ability to perform real-time localization, at over 10 Hz, on a resource-constrained Jetson AGX Orin embedded platform. |
| title | NeRF-VINS: A Real-time Neural Radiance Field Map-based Visual-Inertial Navigation System |
| topic | Robotics |
| url | https://arxiv.org/abs/2309.09295 |