Light Transport-aware Diffusion Posterior Sampling for Single-View Reconstruction of 3D Volumes
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912298458677248 |
|---|---|
| author | Leonard, Ludwic Thuerey, Nils Westermann, Ruediger |
| author_facet | Leonard, Ludwic Thuerey, Nils Westermann, Ruediger |
| contents | We introduce a single-view reconstruction technique of volumetric fields in which multiple light scattering effects are omnipresent, such as in clouds. We model the unknown distribution of volumetric fields using an unconditional diffusion model trained on a novel benchmark dataset comprising 1,000 synthetically simulated volumetric density fields. The neural diffusion model is trained on the latent codes of a novel, diffusion-friendly, monoplanar representation. The generative model is used to incorporate a tailored parametric diffusion posterior sampling technique into different reconstruction tasks. A physically-based differentiable volume renderer is employed to provide gradients with respect to light transport in the latent space. This stands in contrast to classic NeRF approaches and makes the reconstructions better aligned with observed data. Through various experiments, we demonstrate single-view reconstruction of volumetric clouds at a previously unattainable quality. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_05226 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Light Transport-aware Diffusion Posterior Sampling for Single-View Reconstruction of 3D Volumes Leonard, Ludwic Thuerey, Nils Westermann, Ruediger Computer Vision and Pattern Recognition Machine Learning I.2.10; I.4.5 We introduce a single-view reconstruction technique of volumetric fields in which multiple light scattering effects are omnipresent, such as in clouds. We model the unknown distribution of volumetric fields using an unconditional diffusion model trained on a novel benchmark dataset comprising 1,000 synthetically simulated volumetric density fields. The neural diffusion model is trained on the latent codes of a novel, diffusion-friendly, monoplanar representation. The generative model is used to incorporate a tailored parametric diffusion posterior sampling technique into different reconstruction tasks. A physically-based differentiable volume renderer is employed to provide gradients with respect to light transport in the latent space. This stands in contrast to classic NeRF approaches and makes the reconstructions better aligned with observed data. Through various experiments, we demonstrate single-view reconstruction of volumetric clouds at a previously unattainable quality. |
| title | Light Transport-aware Diffusion Posterior Sampling for Single-View Reconstruction of 3D Volumes |
| topic | Computer Vision and Pattern Recognition Machine Learning I.2.10; I.4.5 |
| url | https://arxiv.org/abs/2501.05226 |