NeRF Inpainting with Geometric Diffusion Prior and Balanced Score Distillation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909401305055232 |
|---|---|
| author | Zhang, Menglin Luo, Xin Lan, Yunwei Liu, Chang Li, Rui Zhang, Kaidong Yang, Ganlin Liu, Dong |
| author_facet | Zhang, Menglin Luo, Xin Lan, Yunwei Liu, Chang Li, Rui Zhang, Kaidong Yang, Ganlin Liu, Dong |
| contents | Recent advances in NeRF inpainting have leveraged pretrained diffusion models to enhance performance. However, these methods often yield suboptimal results due to their ineffective utilization of 2D diffusion priors. The limitations manifest in two critical aspects: the inadequate capture of geometric information by pretrained diffusion models and the suboptimal guidance provided by existing Score Distillation Sampling (SDS) methods. To address these problems, we introduce GB-NeRF, a novel framework that enhances NeRF inpainting through improved utilization of 2D diffusion priors. Our approach incorporates two key innovations: a fine-tuning strategy that simultaneously learns appearance and geometric priors and a specialized normal distillation loss that integrates these geometric priors into NeRF inpainting. We propose a technique called Balanced Score Distillation (BSD) that surpasses existing methods such as Score Distillation (SDS) and the improved version, Conditional Score Distillation (CSD). BSD offers improved inpainting quality in appearance and geometric aspects. Extensive experiments show that our method provides superior appearance fidelity and geometric consistency compared to existing approaches. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_15551 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | NeRF Inpainting with Geometric Diffusion Prior and Balanced Score Distillation Zhang, Menglin Luo, Xin Lan, Yunwei Liu, Chang Li, Rui Zhang, Kaidong Yang, Ganlin Liu, Dong Computer Vision and Pattern Recognition Recent advances in NeRF inpainting have leveraged pretrained diffusion models to enhance performance. However, these methods often yield suboptimal results due to their ineffective utilization of 2D diffusion priors. The limitations manifest in two critical aspects: the inadequate capture of geometric information by pretrained diffusion models and the suboptimal guidance provided by existing Score Distillation Sampling (SDS) methods. To address these problems, we introduce GB-NeRF, a novel framework that enhances NeRF inpainting through improved utilization of 2D diffusion priors. Our approach incorporates two key innovations: a fine-tuning strategy that simultaneously learns appearance and geometric priors and a specialized normal distillation loss that integrates these geometric priors into NeRF inpainting. We propose a technique called Balanced Score Distillation (BSD) that surpasses existing methods such as Score Distillation (SDS) and the improved version, Conditional Score Distillation (CSD). BSD offers improved inpainting quality in appearance and geometric aspects. Extensive experiments show that our method provides superior appearance fidelity and geometric consistency compared to existing approaches. |
| title | NeRF Inpainting with Geometric Diffusion Prior and Balanced Score Distillation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2411.15551 |