NEAT: Distilling 3D Wireframes from Neural Attraction Fields
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913296157769728 |
|---|---|
| author | Xue, Nan Tan, Bin Xiao, Yuxi Dong, Liang Xia, Gui-Song Wu, Tianfu Shen, Yujun |
| author_facet | Xue, Nan Tan, Bin Xiao, Yuxi Dong, Liang Xia, Gui-Song Wu, Tianfu Shen, Yujun |
| contents | This paper studies the problem of structured 3D reconstruction using wireframes that consist of line segments and junctions, focusing on the computation of structured boundary geometries of scenes. Instead of leveraging matching-based solutions from 2D wireframes (or line segments) for 3D wireframe reconstruction as done in prior arts, we present NEAT, a rendering-distilling formulation using neural fields to represent 3D line segments with 2D observations, and bipartite matching for perceiving and distilling of a sparse set of 3D global junctions. The proposed {NEAT} enjoys the joint optimization of the neural fields and the global junctions from scratch, using view-dependent 2D observations without precomputed cross-view feature matching. Comprehensive experiments on the DTU and BlendedMVS datasets demonstrate our NEAT's superiority over state-of-the-art alternatives for 3D wireframe reconstruction. Moreover, the distilled 3D global junctions by NEAT, are a better initialization than SfM points, for the recently-emerged 3D Gaussian Splatting for high-fidelity novel view synthesis using about 20 times fewer initial 3D points. Project page: \url{https://xuenan.net/neat}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2307_10206 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | NEAT: Distilling 3D Wireframes from Neural Attraction Fields Xue, Nan Tan, Bin Xiao, Yuxi Dong, Liang Xia, Gui-Song Wu, Tianfu Shen, Yujun Computer Vision and Pattern Recognition Graphics This paper studies the problem of structured 3D reconstruction using wireframes that consist of line segments and junctions, focusing on the computation of structured boundary geometries of scenes. Instead of leveraging matching-based solutions from 2D wireframes (or line segments) for 3D wireframe reconstruction as done in prior arts, we present NEAT, a rendering-distilling formulation using neural fields to represent 3D line segments with 2D observations, and bipartite matching for perceiving and distilling of a sparse set of 3D global junctions. The proposed {NEAT} enjoys the joint optimization of the neural fields and the global junctions from scratch, using view-dependent 2D observations without precomputed cross-view feature matching. Comprehensive experiments on the DTU and BlendedMVS datasets demonstrate our NEAT's superiority over state-of-the-art alternatives for 3D wireframe reconstruction. Moreover, the distilled 3D global junctions by NEAT, are a better initialization than SfM points, for the recently-emerged 3D Gaussian Splatting for high-fidelity novel view synthesis using about 20 times fewer initial 3D points. Project page: \url{https://xuenan.net/neat}. |
| title | NEAT: Distilling 3D Wireframes from Neural Attraction Fields |
| topic | Computer Vision and Pattern Recognition Graphics |
| url | https://arxiv.org/abs/2307.10206 |