UniPlane: Unified Plane Detection and Reconstruction from Posed Monocular Videos
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913416443068416 |
|---|---|
| author | Huang, Yuzhong Liu, Chen Hou, Ji Huo, Ke Dong, Shiyu Morstatter, Fred |
| author_facet | Huang, Yuzhong Liu, Chen Hou, Ji Huo, Ke Dong, Shiyu Morstatter, Fred |
| contents | We present UniPlane, a novel method that unifies plane detection and reconstruction from posed monocular videos. Unlike existing methods that detect planes from local observations and associate them across the video for the final reconstruction, UniPlane unifies both the detection and the reconstruction tasks in a single network, which allows us to directly optimize final reconstruction quality and fully leverage temporal information. Specifically, we build a Transformers-based deep neural network that jointly constructs a 3D feature volume for the environment and estimates a set of per-plane embeddings as queries. UniPlane directly reconstructs the 3D planes by taking dot products between voxel embeddings and the plane embeddings followed by binary thresholding. Extensive experiments on real-world datasets demonstrate that UniPlane outperforms state-of-the-art methods in both plane detection and reconstruction tasks, achieving +4.6 in F-score in geometry as well as consistent improvements in other geometry and segmentation metrics. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_03594 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | UniPlane: Unified Plane Detection and Reconstruction from Posed Monocular Videos Huang, Yuzhong Liu, Chen Hou, Ji Huo, Ke Dong, Shiyu Morstatter, Fred Computer Vision and Pattern Recognition We present UniPlane, a novel method that unifies plane detection and reconstruction from posed monocular videos. Unlike existing methods that detect planes from local observations and associate them across the video for the final reconstruction, UniPlane unifies both the detection and the reconstruction tasks in a single network, which allows us to directly optimize final reconstruction quality and fully leverage temporal information. Specifically, we build a Transformers-based deep neural network that jointly constructs a 3D feature volume for the environment and estimates a set of per-plane embeddings as queries. UniPlane directly reconstructs the 3D planes by taking dot products between voxel embeddings and the plane embeddings followed by binary thresholding. Extensive experiments on real-world datasets demonstrate that UniPlane outperforms state-of-the-art methods in both plane detection and reconstruction tasks, achieving +4.6 in F-score in geometry as well as consistent improvements in other geometry and segmentation metrics. |
| title | UniPlane: Unified Plane Detection and Reconstruction from Posed Monocular Videos |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2407.03594 |