Retinex-Diffusion: On Controlling Illumination Conditions in Diffusion Models via Retinex Theory
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929442816786432 |
|---|---|
| author | Xing, Xiaoyan Hu, Vincent Tao Metzen, Jan Hendrik Groh, Konrad Karaoglu, Sezer Gevers, Theo |
| author_facet | Xing, Xiaoyan Hu, Vincent Tao Metzen, Jan Hendrik Groh, Konrad Karaoglu, Sezer Gevers, Theo |
| contents | This paper introduces a novel approach to illumination manipulation in diffusion models, addressing the gap in conditional image generation with a focus on lighting conditions. We conceptualize the diffusion model as a black-box image render and strategically decompose its energy function in alignment with the image formation model. Our method effectively separates and controls illumination-related properties during the generative process. It generates images with realistic illumination effects, including cast shadow, soft shadow, and inter-reflections. Remarkably, it achieves this without the necessity for learning intrinsic decomposition, finding directions in latent space, or undergoing additional training with new datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_20785 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Retinex-Diffusion: On Controlling Illumination Conditions in Diffusion Models via Retinex Theory Xing, Xiaoyan Hu, Vincent Tao Metzen, Jan Hendrik Groh, Konrad Karaoglu, Sezer Gevers, Theo Computer Vision and Pattern Recognition This paper introduces a novel approach to illumination manipulation in diffusion models, addressing the gap in conditional image generation with a focus on lighting conditions. We conceptualize the diffusion model as a black-box image render and strategically decompose its energy function in alignment with the image formation model. Our method effectively separates and controls illumination-related properties during the generative process. It generates images with realistic illumination effects, including cast shadow, soft shadow, and inter-reflections. Remarkably, it achieves this without the necessity for learning intrinsic decomposition, finding directions in latent space, or undergoing additional training with new datasets. |
| title | Retinex-Diffusion: On Controlling Illumination Conditions in Diffusion Models via Retinex Theory |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2407.20785 |