Curved Diffusion: A Generative Model With Optical Geometry Control
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911955337347072 |
|---|---|
| author | Voynov, Andrey Hertz, Amir Arar, Moab Fruchter, Shlomi Cohen-Or, Daniel |
| author_facet | Voynov, Andrey Hertz, Amir Arar, Moab Fruchter, Shlomi Cohen-Or, Daniel |
| contents | State-of-the-art diffusion models can generate highly realistic images based on various conditioning like text, segmentation, and depth. However, an essential aspect often overlooked is the specific camera geometry used during image capture. The influence of different optical systems on the final scene appearance is frequently overlooked. This study introduces a framework that intimately integrates a text-to-image diffusion model with the particular lens geometry used in image rendering. Our method is based on a per-pixel coordinate conditioning method, enabling the control over the rendering geometry. Notably, we demonstrate the manipulation of curvature properties, achieving diverse visual effects, such as fish-eye, panoramic views, and spherical texturing using a single diffusion model. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_17609 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Curved Diffusion: A Generative Model With Optical Geometry Control Voynov, Andrey Hertz, Amir Arar, Moab Fruchter, Shlomi Cohen-Or, Daniel Computer Vision and Pattern Recognition Graphics Machine Learning State-of-the-art diffusion models can generate highly realistic images based on various conditioning like text, segmentation, and depth. However, an essential aspect often overlooked is the specific camera geometry used during image capture. The influence of different optical systems on the final scene appearance is frequently overlooked. This study introduces a framework that intimately integrates a text-to-image diffusion model with the particular lens geometry used in image rendering. Our method is based on a per-pixel coordinate conditioning method, enabling the control over the rendering geometry. Notably, we demonstrate the manipulation of curvature properties, achieving diverse visual effects, such as fish-eye, panoramic views, and spherical texturing using a single diffusion model. |
| title | Curved Diffusion: A Generative Model With Optical Geometry Control |
| topic | Computer Vision and Pattern Recognition Graphics Machine Learning |
| url | https://arxiv.org/abs/2311.17609 |