PointNeRF++: A multi-scale, point-based Neural Radiance Field
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909145910738944 |
|---|---|
| author | Sun, Weiwei Trulls, Eduard Tseng, Yang-Che Sambandam, Sneha Sharma, Gopal Tagliasacchi, Andrea Yi, Kwang Moo |
| author_facet | Sun, Weiwei Trulls, Eduard Tseng, Yang-Che Sambandam, Sneha Sharma, Gopal Tagliasacchi, Andrea Yi, Kwang Moo |
| contents | Point clouds offer an attractive source of information to complement images in neural scene representations, especially when few images are available. Neural rendering methods based on point clouds do exist, but they do not perform well when the point cloud quality is low -- e.g., sparse or incomplete, which is often the case with real-world data. We overcome these problems with a simple representation that aggregates point clouds at multiple scale levels with sparse voxel grids at different resolutions. To deal with point cloud sparsity, we average across multiple scale levels -- but only among those that are valid, i.e., that have enough neighboring points in proximity to the ray of a pixel. To help model areas without points, we add a global voxel at the coarsest scale, thus unifying ``classical'' and point-based NeRF formulations. We validate our method on the NeRF Synthetic, ScanNet, and KITTI-360 datasets, outperforming the state of the art, with a significant gap compared to other NeRF-based methods, especially on more challenging scenes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_02362 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | PointNeRF++: A multi-scale, point-based Neural Radiance Field Sun, Weiwei Trulls, Eduard Tseng, Yang-Che Sambandam, Sneha Sharma, Gopal Tagliasacchi, Andrea Yi, Kwang Moo Computer Vision and Pattern Recognition Graphics Point clouds offer an attractive source of information to complement images in neural scene representations, especially when few images are available. Neural rendering methods based on point clouds do exist, but they do not perform well when the point cloud quality is low -- e.g., sparse or incomplete, which is often the case with real-world data. We overcome these problems with a simple representation that aggregates point clouds at multiple scale levels with sparse voxel grids at different resolutions. To deal with point cloud sparsity, we average across multiple scale levels -- but only among those that are valid, i.e., that have enough neighboring points in proximity to the ray of a pixel. To help model areas without points, we add a global voxel at the coarsest scale, thus unifying ``classical'' and point-based NeRF formulations. We validate our method on the NeRF Synthetic, ScanNet, and KITTI-360 datasets, outperforming the state of the art, with a significant gap compared to other NeRF-based methods, especially on more challenging scenes. |
| title | PointNeRF++: A multi-scale, point-based Neural Radiance Field |
| topic | Computer Vision and Pattern Recognition Graphics |
| url | https://arxiv.org/abs/2312.02362 |