Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2405.14847 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914808217993216 |
|---|---|
| author | Wu, Liwen Bi, Sai Xu, Zexiang Luan, Fujun Zhang, Kai Georgiev, Iliyan Sunkavalli, Kalyan Ramamoorthi, Ravi |
| author_facet | Wu, Liwen Bi, Sai Xu, Zexiang Luan, Fujun Zhang, Kai Georgiev, Iliyan Sunkavalli, Kalyan Ramamoorthi, Ravi |
| contents | Novel-view synthesis of specular objects like shiny metals or glossy paints remains a significant challenge. Not only the glossy appearance but also global illumination effects, including reflections of other objects in the environment, are critical components to faithfully reproduce a scene. In this paper, we present Neural Directional Encoding (NDE), a view-dependent appearance encoding of neural radiance fields (NeRF) for rendering specular objects. NDE transfers the concept of feature-grid-based spatial encoding to the angular domain, significantly improving the ability to model high-frequency angular signals. In contrast to previous methods that use encoding functions with only angular input, we additionally cone-trace spatial features to obtain a spatially varying directional encoding, which addresses the challenging interreflection effects. Extensive experiments on both synthetic and real datasets show that a NeRF model with NDE (1) outperforms the state of the art on view synthesis of specular objects, and (2) works with small networks to allow fast (real-time) inference. The project webpage and source code are available at: \url{https://lwwu2.github.io/nde/}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_14847 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Neural Directional Encoding for Efficient and Accurate View-Dependent Appearance Modeling Wu, Liwen Bi, Sai Xu, Zexiang Luan, Fujun Zhang, Kai Georgiev, Iliyan Sunkavalli, Kalyan Ramamoorthi, Ravi Computer Vision and Pattern Recognition Novel-view synthesis of specular objects like shiny metals or glossy paints remains a significant challenge. Not only the glossy appearance but also global illumination effects, including reflections of other objects in the environment, are critical components to faithfully reproduce a scene. In this paper, we present Neural Directional Encoding (NDE), a view-dependent appearance encoding of neural radiance fields (NeRF) for rendering specular objects. NDE transfers the concept of feature-grid-based spatial encoding to the angular domain, significantly improving the ability to model high-frequency angular signals. In contrast to previous methods that use encoding functions with only angular input, we additionally cone-trace spatial features to obtain a spatially varying directional encoding, which addresses the challenging interreflection effects. Extensive experiments on both synthetic and real datasets show that a NeRF model with NDE (1) outperforms the state of the art on view synthesis of specular objects, and (2) works with small networks to allow fast (real-time) inference. The project webpage and source code are available at: \url{https://lwwu2.github.io/nde/}. |
| title | Neural Directional Encoding for Efficient and Accurate View-Dependent Appearance Modeling |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2405.14847 |