MonoPatchNeRF: Improving Neural Radiance Fields with Patch-based Monocular Guidance
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914920106295296 |
|---|---|
| author | Wu, Yuqun Lee, Jae Yong Zou, Chuhang Wang, Shenlong Hoiem, Derek |
| author_facet | Wu, Yuqun Lee, Jae Yong Zou, Chuhang Wang, Shenlong Hoiem, Derek |
| contents | The latest regularized Neural Radiance Field (NeRF) approaches produce poor geometry and view extrapolation for large scale sparse view scenes, such as ETH3D. Density-based approaches tend to be under-constrained, while surface-based approaches tend to miss details. In this paper, we take a density-based approach, sampling patches instead of individual rays to better incorporate monocular depth and normal estimates and patch-based photometric consistency constraints between training views and sampled virtual views. Loosely constraining densities based on estimated depth aligned to sparse points further improves geometric accuracy. While maintaining similar view synthesis quality, our approach significantly improves geometric accuracy on the ETH3D benchmark, e.g. increasing the F1@2cm score by 4x-8x compared to other regularized density-based approaches, with much lower training and inference time than other approaches. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_08252 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MonoPatchNeRF: Improving Neural Radiance Fields with Patch-based Monocular Guidance Wu, Yuqun Lee, Jae Yong Zou, Chuhang Wang, Shenlong Hoiem, Derek Computer Vision and Pattern Recognition The latest regularized Neural Radiance Field (NeRF) approaches produce poor geometry and view extrapolation for large scale sparse view scenes, such as ETH3D. Density-based approaches tend to be under-constrained, while surface-based approaches tend to miss details. In this paper, we take a density-based approach, sampling patches instead of individual rays to better incorporate monocular depth and normal estimates and patch-based photometric consistency constraints between training views and sampled virtual views. Loosely constraining densities based on estimated depth aligned to sparse points further improves geometric accuracy. While maintaining similar view synthesis quality, our approach significantly improves geometric accuracy on the ETH3D benchmark, e.g. increasing the F1@2cm score by 4x-8x compared to other regularized density-based approaches, with much lower training and inference time than other approaches. |
| title | MonoPatchNeRF: Improving Neural Radiance Fields with Patch-based Monocular Guidance |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2404.08252 |