UniRelight: Learning Joint Decomposition and Synthesis for Video Relighting
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912438591422464 |
|---|---|
| author | He, Kai Liang, Ruofan Munkberg, Jacob Hasselgren, Jon Vijaykumar, Nandita Keller, Alexander Fidler, Sanja Gilitschenski, Igor Gojcic, Zan Wang, Zian |
| author_facet | He, Kai Liang, Ruofan Munkberg, Jacob Hasselgren, Jon Vijaykumar, Nandita Keller, Alexander Fidler, Sanja Gilitschenski, Igor Gojcic, Zan Wang, Zian |
| contents | We address the challenge of relighting a single image or video, a task that demands precise scene intrinsic understanding and high-quality light transport synthesis. Existing end-to-end relighting models are often limited by the scarcity of paired multi-illumination data, restricting their ability to generalize across diverse scenes. Conversely, two-stage pipelines that combine inverse and forward rendering can mitigate data requirements but are susceptible to error accumulation and often fail to produce realistic outputs under complex lighting conditions or with sophisticated materials. In this work, we introduce a general-purpose approach that jointly estimates albedo and synthesizes relit outputs in a single pass, harnessing the generative capabilities of video diffusion models. This joint formulation enhances implicit scene comprehension and facilitates the creation of realistic lighting effects and intricate material interactions, such as shadows, reflections, and transparency. Trained on synthetic multi-illumination data and extensive automatically labeled real-world videos, our model demonstrates strong generalization across diverse domains and surpasses previous methods in both visual fidelity and temporal consistency. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_15673 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | UniRelight: Learning Joint Decomposition and Synthesis for Video Relighting He, Kai Liang, Ruofan Munkberg, Jacob Hasselgren, Jon Vijaykumar, Nandita Keller, Alexander Fidler, Sanja Gilitschenski, Igor Gojcic, Zan Wang, Zian Computer Vision and Pattern Recognition We address the challenge of relighting a single image or video, a task that demands precise scene intrinsic understanding and high-quality light transport synthesis. Existing end-to-end relighting models are often limited by the scarcity of paired multi-illumination data, restricting their ability to generalize across diverse scenes. Conversely, two-stage pipelines that combine inverse and forward rendering can mitigate data requirements but are susceptible to error accumulation and often fail to produce realistic outputs under complex lighting conditions or with sophisticated materials. In this work, we introduce a general-purpose approach that jointly estimates albedo and synthesizes relit outputs in a single pass, harnessing the generative capabilities of video diffusion models. This joint formulation enhances implicit scene comprehension and facilitates the creation of realistic lighting effects and intricate material interactions, such as shadows, reflections, and transparency. Trained on synthetic multi-illumination data and extensive automatically labeled real-world videos, our model demonstrates strong generalization across diverse domains and surpasses previous methods in both visual fidelity and temporal consistency. |
| title | UniRelight: Learning Joint Decomposition and Synthesis for Video Relighting |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2506.15673 |