WonderPlay: Dynamic 3D Scene Generation from a Single Image and Actions
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917112959729664 |
|---|---|
| author | Li, Zizhang Yu, Hong-Xing Liu, Wei Yang, Yin Herrmann, Charles Wetzstein, Gordon Wu, Jiajun |
| author_facet | Li, Zizhang Yu, Hong-Xing Liu, Wei Yang, Yin Herrmann, Charles Wetzstein, Gordon Wu, Jiajun |
| contents | WonderPlay is a novel framework integrating physics simulation with video generation for generating action-conditioned dynamic 3D scenes from a single image. While prior works are restricted to rigid body or simple elastic dynamics, WonderPlay features a hybrid generative simulator to synthesize a wide range of 3D dynamics. The hybrid generative simulator first uses a physics solver to simulate coarse 3D dynamics, which subsequently conditions a video generator to produce a video with finer, more realistic motion. The generated video is then used to update the simulated dynamic 3D scene, closing the loop between the physics solver and the video generator. This approach enables intuitive user control to be combined with the accurate dynamics of physics-based simulators and the expressivity of diffusion-based video generators. Experimental results demonstrate that WonderPlay enables users to interact with various scenes of diverse content, including cloth, sand, snow, liquid, smoke, elastic, and rigid bodies -- all using a single image input. Code will be made public. Project website: https://kyleleey.github.io/WonderPlay/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_18151 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | WonderPlay: Dynamic 3D Scene Generation from a Single Image and Actions Li, Zizhang Yu, Hong-Xing Liu, Wei Yang, Yin Herrmann, Charles Wetzstein, Gordon Wu, Jiajun Graphics Artificial Intelligence Computer Vision and Pattern Recognition WonderPlay is a novel framework integrating physics simulation with video generation for generating action-conditioned dynamic 3D scenes from a single image. While prior works are restricted to rigid body or simple elastic dynamics, WonderPlay features a hybrid generative simulator to synthesize a wide range of 3D dynamics. The hybrid generative simulator first uses a physics solver to simulate coarse 3D dynamics, which subsequently conditions a video generator to produce a video with finer, more realistic motion. The generated video is then used to update the simulated dynamic 3D scene, closing the loop between the physics solver and the video generator. This approach enables intuitive user control to be combined with the accurate dynamics of physics-based simulators and the expressivity of diffusion-based video generators. Experimental results demonstrate that WonderPlay enables users to interact with various scenes of diverse content, including cloth, sand, snow, liquid, smoke, elastic, and rigid bodies -- all using a single image input. Code will be made public. Project website: https://kyleleey.github.io/WonderPlay/ |
| title | WonderPlay: Dynamic 3D Scene Generation from a Single Image and Actions |
| topic | Graphics Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2505.18151 |