NeRF Revisited: Fixing Quadrature Instability in Volume Rendering
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914646209855488 |
|---|---|
| author | Uy, Mikaela Angelina Nakayama, Kiyohiro Yang, Guandao Thomas, Rahul Krishna Guibas, Leonidas Li, Ke |
| author_facet | Uy, Mikaela Angelina Nakayama, Kiyohiro Yang, Guandao Thomas, Rahul Krishna Guibas, Leonidas Li, Ke |
| contents | Neural radiance fields (NeRF) rely on volume rendering to synthesize novel views. Volume rendering requires evaluating an integral along each ray, which is numerically approximated with a finite sum that corresponds to the exact integral along the ray under piecewise constant volume density. As a consequence, the rendered result is unstable w.r.t. the choice of samples along the ray, a phenomenon that we dub quadrature instability. We propose a mathematically principled solution by reformulating the sample-based rendering equation so that it corresponds to the exact integral under piecewise linear volume density. This simultaneously resolves multiple issues: conflicts between samples along different rays, imprecise hierarchical sampling, and non-differentiability of quantiles of ray termination distances w.r.t. model parameters. We demonstrate several benefits over the classical sample-based rendering equation, such as sharper textures, better geometric reconstruction, and stronger depth supervision. Our proposed formulation can be also be used as a drop-in replacement to the volume rendering equation of existing NeRF-based methods. Our project page can be found at pl-nerf.github.io. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_20685 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | NeRF Revisited: Fixing Quadrature Instability in Volume Rendering Uy, Mikaela Angelina Nakayama, Kiyohiro Yang, Guandao Thomas, Rahul Krishna Guibas, Leonidas Li, Ke Computer Vision and Pattern Recognition Neural radiance fields (NeRF) rely on volume rendering to synthesize novel views. Volume rendering requires evaluating an integral along each ray, which is numerically approximated with a finite sum that corresponds to the exact integral along the ray under piecewise constant volume density. As a consequence, the rendered result is unstable w.r.t. the choice of samples along the ray, a phenomenon that we dub quadrature instability. We propose a mathematically principled solution by reformulating the sample-based rendering equation so that it corresponds to the exact integral under piecewise linear volume density. This simultaneously resolves multiple issues: conflicts between samples along different rays, imprecise hierarchical sampling, and non-differentiability of quantiles of ray termination distances w.r.t. model parameters. We demonstrate several benefits over the classical sample-based rendering equation, such as sharper textures, better geometric reconstruction, and stronger depth supervision. Our proposed formulation can be also be used as a drop-in replacement to the volume rendering equation of existing NeRF-based methods. Our project page can be found at pl-nerf.github.io. |
| title | NeRF Revisited: Fixing Quadrature Instability in Volume Rendering |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2310.20685 |