DATENeRF: Depth-Aware Text-based Editing of NeRFs
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910549940371456 |
|---|---|
| author | Rojas, Sara Philip, Julien Zhang, Kai Bi, Sai Luan, Fujun Ghanem, Bernard Sunkavall, Kalyan |
| author_facet | Rojas, Sara Philip, Julien Zhang, Kai Bi, Sai Luan, Fujun Ghanem, Bernard Sunkavall, Kalyan |
| contents | Recent advancements in diffusion models have shown remarkable proficiency in editing 2D images based on text prompts. However, extending these techniques to edit scenes in Neural Radiance Fields (NeRF) is complex, as editing individual 2D frames can result in inconsistencies across multiple views. Our crucial insight is that a NeRF scene's geometry can serve as a bridge to integrate these 2D edits. Utilizing this geometry, we employ a depth-conditioned ControlNet to enhance the coherence of each 2D image modification. Moreover, we introduce an inpainting approach that leverages the depth information of NeRF scenes to distribute 2D edits across different images, ensuring robustness against errors and resampling challenges. Our results reveal that this methodology achieves more consistent, lifelike, and detailed edits than existing leading methods for text-driven NeRF scene editing. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_04526 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | DATENeRF: Depth-Aware Text-based Editing of NeRFs Rojas, Sara Philip, Julien Zhang, Kai Bi, Sai Luan, Fujun Ghanem, Bernard Sunkavall, Kalyan Computer Vision and Pattern Recognition Recent advancements in diffusion models have shown remarkable proficiency in editing 2D images based on text prompts. However, extending these techniques to edit scenes in Neural Radiance Fields (NeRF) is complex, as editing individual 2D frames can result in inconsistencies across multiple views. Our crucial insight is that a NeRF scene's geometry can serve as a bridge to integrate these 2D edits. Utilizing this geometry, we employ a depth-conditioned ControlNet to enhance the coherence of each 2D image modification. Moreover, we introduce an inpainting approach that leverages the depth information of NeRF scenes to distribute 2D edits across different images, ensuring robustness against errors and resampling challenges. Our results reveal that this methodology achieves more consistent, lifelike, and detailed edits than existing leading methods for text-driven NeRF scene editing. |
| title | DATENeRF: Depth-Aware Text-based Editing of NeRFs |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2404.04526 |