DreamDrone: Text-to-Image Diffusion Models are Zero-shot Perpetual View Generators
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914955474763776 |
|---|---|
| author | Kong, Hanyang Lian, Dongze Mi, Michael Bi Wang, Xinchao |
| author_facet | Kong, Hanyang Lian, Dongze Mi, Michael Bi Wang, Xinchao |
| contents | We introduce DreamDrone, a novel zero-shot and training-free pipeline for generating unbounded flythrough scenes from textual prompts. Different from other methods that focus on warping images frame by frame, we advocate explicitly warping the intermediate latent code of the pre-trained text-to-image diffusion model for high-quality image generation and generalization ability. To further enhance the fidelity of the generated images, we also propose a feature-correspondence-guidance diffusion process and a high-pass filtering strategy to promote geometric consistency and high-frequency detail consistency, respectively. Extensive experiments reveal that DreamDrone significantly surpasses existing methods, delivering highly authentic scene generation with exceptional visual quality, without training or fine-tuning on datasets or reconstructing 3D point clouds in advance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_08746 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | DreamDrone: Text-to-Image Diffusion Models are Zero-shot Perpetual View Generators Kong, Hanyang Lian, Dongze Mi, Michael Bi Wang, Xinchao Computer Vision and Pattern Recognition We introduce DreamDrone, a novel zero-shot and training-free pipeline for generating unbounded flythrough scenes from textual prompts. Different from other methods that focus on warping images frame by frame, we advocate explicitly warping the intermediate latent code of the pre-trained text-to-image diffusion model for high-quality image generation and generalization ability. To further enhance the fidelity of the generated images, we also propose a feature-correspondence-guidance diffusion process and a high-pass filtering strategy to promote geometric consistency and high-frequency detail consistency, respectively. Extensive experiments reveal that DreamDrone significantly surpasses existing methods, delivering highly authentic scene generation with exceptional visual quality, without training or fine-tuning on datasets or reconstructing 3D point clouds in advance. |
| title | DreamDrone: Text-to-Image Diffusion Models are Zero-shot Perpetual View Generators |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2312.08746 |