ViewFusion: Towards Multi-View Consistency via Interpolated Denoising
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929259232100352 |
|---|---|
| author | Yang, Xianghui Zuo, Yan Ramasinghe, Sameera Bazzani, Loris Avraham, Gil Hengel, Anton van den |
| author_facet | Yang, Xianghui Zuo, Yan Ramasinghe, Sameera Bazzani, Loris Avraham, Gil Hengel, Anton van den |
| contents | Novel-view synthesis through diffusion models has demonstrated remarkable potential for generating diverse and high-quality images. Yet, the independent process of image generation in these prevailing methods leads to challenges in maintaining multiple-view consistency. To address this, we introduce ViewFusion, a novel, training-free algorithm that can be seamlessly integrated into existing pre-trained diffusion models. Our approach adopts an auto-regressive method that implicitly leverages previously generated views as context for the next view generation, ensuring robust multi-view consistency during the novel-view generation process. Through a diffusion process that fuses known-view information via interpolated denoising, our framework successfully extends single-view conditioned models to work in multiple-view conditional settings without any additional fine-tuning. Extensive experimental results demonstrate the effectiveness of ViewFusion in generating consistent and detailed novel views. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_18842 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ViewFusion: Towards Multi-View Consistency via Interpolated Denoising Yang, Xianghui Zuo, Yan Ramasinghe, Sameera Bazzani, Loris Avraham, Gil Hengel, Anton van den Computer Vision and Pattern Recognition Novel-view synthesis through diffusion models has demonstrated remarkable potential for generating diverse and high-quality images. Yet, the independent process of image generation in these prevailing methods leads to challenges in maintaining multiple-view consistency. To address this, we introduce ViewFusion, a novel, training-free algorithm that can be seamlessly integrated into existing pre-trained diffusion models. Our approach adopts an auto-regressive method that implicitly leverages previously generated views as context for the next view generation, ensuring robust multi-view consistency during the novel-view generation process. Through a diffusion process that fuses known-view information via interpolated denoising, our framework successfully extends single-view conditioned models to work in multiple-view conditional settings without any additional fine-tuning. Extensive experimental results demonstrate the effectiveness of ViewFusion in generating consistent and detailed novel views. |
| title | ViewFusion: Towards Multi-View Consistency via Interpolated Denoising |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2402.18842 |