Beyond Visual Field of View: Perceiving 3D Environment with Echoes and Vision
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2022
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913228154470400 |
|---|---|
| author | Zhu, Lingyu Rahtu, Esa Zhao, Hang |
| author_facet | Zhu, Lingyu Rahtu, Esa Zhao, Hang |
| contents | This paper focuses on perceiving and navigating 3D environments using echoes and RGB image. In particular, we perform depth estimation by fusing RGB image with echoes, received from multiple orientations. Unlike previous works, we go beyond the field of view of the RGB and estimate dense depth maps for substantially larger parts of the environment. We show that the echoes provide holistic and in-expensive information about the 3D structures complementing the RGB image. Moreover, we study how echoes and the wide field-of-view depth maps can be utilised in robot navigation. We compare the proposed methods against recent baselines using two sets of challenging realistic 3D environments: Replica and Matterport3D. The implementation and pre-trained models will be made publicly available. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2207_01136 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Beyond Visual Field of View: Perceiving 3D Environment with Echoes and Vision Zhu, Lingyu Rahtu, Esa Zhao, Hang Computer Vision and Pattern Recognition This paper focuses on perceiving and navigating 3D environments using echoes and RGB image. In particular, we perform depth estimation by fusing RGB image with echoes, received from multiple orientations. Unlike previous works, we go beyond the field of view of the RGB and estimate dense depth maps for substantially larger parts of the environment. We show that the echoes provide holistic and in-expensive information about the 3D structures complementing the RGB image. Moreover, we study how echoes and the wide field-of-view depth maps can be utilised in robot navigation. We compare the proposed methods against recent baselines using two sets of challenging realistic 3D environments: Replica and Matterport3D. The implementation and pre-trained models will be made publicly available. |
| title | Beyond Visual Field of View: Perceiving 3D Environment with Echoes and Vision |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2207.01136 |