Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2406.10543 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914835133890560 |
|---|---|
| author | Tang, Zhenggang Ren, Zhongzheng Zhao, Xiaoming Wen, Bowen Tremblay, Jonathan Birchfield, Stan Schwing, Alexander |
| author_facet | Tang, Zhenggang Ren, Zhongzheng Zhao, Xiaoming Wen, Bowen Tremblay, Jonathan Birchfield, Stan Schwing, Alexander |
| contents | We present a method for automatically modifying a NeRF representation based on a single observation of a non-rigid transformed version of the original scene. Our method defines the transformation as a 3D flow, specifically as a weighted linear blending of rigid transformations of 3D anchor points that are defined on the surface of the scene. In order to identify anchor points, we introduce a novel correspondence algorithm that first matches RGB-based pairs, then leverages multi-view information and 3D reprojection to robustly filter false positives in two steps. We also introduce a new dataset for exploring the problem of modifying a NeRF scene through a single observation. Our dataset ( https://github.com/nerfdeformer/nerfdeformer ) contains 113 synthetic scenes leveraging 47 3D assets. We show that our proposed method outperforms NeRF editing methods as well as diffusion-based methods, and we also explore different methods for filtering correspondences. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_10543 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | NeRFDeformer: NeRF Transformation from a Single View via 3D Scene Flows Tang, Zhenggang Ren, Zhongzheng Zhao, Xiaoming Wen, Bowen Tremblay, Jonathan Birchfield, Stan Schwing, Alexander Computer Vision and Pattern Recognition Artificial Intelligence We present a method for automatically modifying a NeRF representation based on a single observation of a non-rigid transformed version of the original scene. Our method defines the transformation as a 3D flow, specifically as a weighted linear blending of rigid transformations of 3D anchor points that are defined on the surface of the scene. In order to identify anchor points, we introduce a novel correspondence algorithm that first matches RGB-based pairs, then leverages multi-view information and 3D reprojection to robustly filter false positives in two steps. We also introduce a new dataset for exploring the problem of modifying a NeRF scene through a single observation. Our dataset ( https://github.com/nerfdeformer/nerfdeformer ) contains 113 synthetic scenes leveraging 47 3D assets. We show that our proposed method outperforms NeRF editing methods as well as diffusion-based methods, and we also explore different methods for filtering correspondences. |
| title | NeRFDeformer: NeRF Transformation from a Single View via 3D Scene Flows |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2406.10543 |