CaesarNeRF: Calibrated Semantic Representation for Few-shot Generalizable Neural Rendering
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909249063354368 |
|---|---|
| author | Zhu, Haidong Ding, Tianyu Chen, Tianyi Zharkov, Ilya Nevatia, Ram Liang, Luming |
| author_facet | Zhu, Haidong Ding, Tianyu Chen, Tianyi Zharkov, Ilya Nevatia, Ram Liang, Luming |
| contents | Generalizability and few-shot learning are key challenges in Neural Radiance Fields (NeRF), often due to the lack of a holistic understanding in pixel-level rendering. We introduce CaesarNeRF, an end-to-end approach that leverages scene-level CAlibratEd SemAntic Representation along with pixel-level representations to advance few-shot, generalizable neural rendering, facilitating a holistic understanding without compromising high-quality details. CaesarNeRF explicitly models pose differences of reference views to combine scene-level semantic representations, providing a calibrated holistic understanding. This calibration process aligns various viewpoints with precise location and is further enhanced by sequential refinement to capture varying details. Extensive experiments on public datasets, including LLFF, Shiny, mip-NeRF 360, and MVImgNet, show that CaesarNeRF delivers state-of-the-art performance across varying numbers of reference views, proving effective even with a single reference image. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_15510 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | CaesarNeRF: Calibrated Semantic Representation for Few-shot Generalizable Neural Rendering Zhu, Haidong Ding, Tianyu Chen, Tianyi Zharkov, Ilya Nevatia, Ram Liang, Luming Computer Vision and Pattern Recognition Generalizability and few-shot learning are key challenges in Neural Radiance Fields (NeRF), often due to the lack of a holistic understanding in pixel-level rendering. We introduce CaesarNeRF, an end-to-end approach that leverages scene-level CAlibratEd SemAntic Representation along with pixel-level representations to advance few-shot, generalizable neural rendering, facilitating a holistic understanding without compromising high-quality details. CaesarNeRF explicitly models pose differences of reference views to combine scene-level semantic representations, providing a calibrated holistic understanding. This calibration process aligns various viewpoints with precise location and is further enhanced by sequential refinement to capture varying details. Extensive experiments on public datasets, including LLFF, Shiny, mip-NeRF 360, and MVImgNet, show that CaesarNeRF delivers state-of-the-art performance across varying numbers of reference views, proving effective even with a single reference image. |
| title | CaesarNeRF: Calibrated Semantic Representation for Few-shot Generalizable Neural Rendering |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2311.15510 |