SViM3D: Stable Video Material Diffusion for Single Image 3D Generation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909881538183168 |
|---|---|
| author | Engelhardt, Andreas Boss, Mark Voleti, Vikram Yao, Chun-Han Lensch, Hendrik P. A. Jampani, Varun |
| author_facet | Engelhardt, Andreas Boss, Mark Voleti, Vikram Yao, Chun-Han Lensch, Hendrik P. A. Jampani, Varun |
| contents | We present Stable Video Materials 3D (SViM3D), a framework to predict multi-view consistent physically based rendering (PBR) materials, given a single image. Recently, video diffusion models have been successfully used to reconstruct 3D objects from a single image efficiently. However, reflectance is still represented by simple material models or needs to be estimated in additional steps to enable relighting and controlled appearance edits. We extend a latent video diffusion model to output spatially varying PBR parameters and surface normals jointly with each generated view based on explicit camera control. This unique setup allows for relighting and generating a 3D asset using our model as neural prior. We introduce various mechanisms to this pipeline that improve quality in this ill-posed setting. We show state-of-the-art relighting and novel view synthesis performance on multiple object-centric datasets. Our method generalizes to diverse inputs, enabling the generation of relightable 3D assets useful in AR/VR, movies, games and other visual media. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_08271 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SViM3D: Stable Video Material Diffusion for Single Image 3D Generation Engelhardt, Andreas Boss, Mark Voleti, Vikram Yao, Chun-Han Lensch, Hendrik P. A. Jampani, Varun Graphics Computer Vision and Pattern Recognition Machine Learning We present Stable Video Materials 3D (SViM3D), a framework to predict multi-view consistent physically based rendering (PBR) materials, given a single image. Recently, video diffusion models have been successfully used to reconstruct 3D objects from a single image efficiently. However, reflectance is still represented by simple material models or needs to be estimated in additional steps to enable relighting and controlled appearance edits. We extend a latent video diffusion model to output spatially varying PBR parameters and surface normals jointly with each generated view based on explicit camera control. This unique setup allows for relighting and generating a 3D asset using our model as neural prior. We introduce various mechanisms to this pipeline that improve quality in this ill-posed setting. We show state-of-the-art relighting and novel view synthesis performance on multiple object-centric datasets. Our method generalizes to diverse inputs, enabling the generation of relightable 3D assets useful in AR/VR, movies, games and other visual media. |
| title | SViM3D: Stable Video Material Diffusion for Single Image 3D Generation |
| topic | Graphics Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2510.08271 |