SynthLight: Portrait Relighting with Diffusion Model by Learning to Re-render Synthetic Faces
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916568851546112 |
|---|---|
| author | Chaturvedi, Sumit Ren, Mengwei Hold-Geoffroy, Yannick Liu, Jingyuan Dorsey, Julie Shu, Zhixin |
| author_facet | Chaturvedi, Sumit Ren, Mengwei Hold-Geoffroy, Yannick Liu, Jingyuan Dorsey, Julie Shu, Zhixin |
| contents | We introduce SynthLight, a diffusion model for portrait relighting. Our approach frames image relighting as a re-rendering problem, where pixels are transformed in response to changes in environmental lighting conditions. Using a physically-based rendering engine, we synthesize a dataset to simulate this lighting-conditioned transformation with 3D head assets under varying lighting. We propose two training and inference strategies to bridge the gap between the synthetic and real image domains: (1) multi-task training that takes advantage of real human portraits without lighting labels; (2) an inference time diffusion sampling procedure based on classifier-free guidance that leverages the input portrait to better preserve details. Our method generalizes to diverse real photographs and produces realistic illumination effects, including specular highlights and cast shadows, while preserving the subject's identity. Our quantitative experiments on Light Stage data demonstrate results comparable to state-of-the-art relighting methods. Our qualitative results on in-the-wild images showcase rich and unprecedented illumination effects. Project Page: \url{https://vrroom.github.io/synthlight/} |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_09756 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SynthLight: Portrait Relighting with Diffusion Model by Learning to Re-render Synthetic Faces Chaturvedi, Sumit Ren, Mengwei Hold-Geoffroy, Yannick Liu, Jingyuan Dorsey, Julie Shu, Zhixin Computer Vision and Pattern Recognition Graphics We introduce SynthLight, a diffusion model for portrait relighting. Our approach frames image relighting as a re-rendering problem, where pixels are transformed in response to changes in environmental lighting conditions. Using a physically-based rendering engine, we synthesize a dataset to simulate this lighting-conditioned transformation with 3D head assets under varying lighting. We propose two training and inference strategies to bridge the gap between the synthetic and real image domains: (1) multi-task training that takes advantage of real human portraits without lighting labels; (2) an inference time diffusion sampling procedure based on classifier-free guidance that leverages the input portrait to better preserve details. Our method generalizes to diverse real photographs and produces realistic illumination effects, including specular highlights and cast shadows, while preserving the subject's identity. Our quantitative experiments on Light Stage data demonstrate results comparable to state-of-the-art relighting methods. Our qualitative results on in-the-wild images showcase rich and unprecedented illumination effects. Project Page: \url{https://vrroom.github.io/synthlight/} |
| title | SynthLight: Portrait Relighting with Diffusion Model by Learning to Re-render Synthetic Faces |
| topic | Computer Vision and Pattern Recognition Graphics |
| url | https://arxiv.org/abs/2501.09756 |