Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2511.21978 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913056021282816 |
|---|---|
| author | Lin, Guying Huang, Kemeng Liu, Michael Gao, Ruihan Chen, Hanke Chen, Lyuhao Lu, Beijia Komura, Taku Liu, Yuan Zhu, Jun-Yan Li, Minchen |
| author_facet | Lin, Guying Huang, Kemeng Liu, Michael Gao, Ruihan Chen, Hanke Chen, Lyuhao Lu, Beijia Komura, Taku Liu, Yuan Zhu, Jun-Yan Li, Minchen |
| contents | We introduce PAT3D, the first physics-augmented text-to-3D scene generation framework that integrates vision-language models with physics-based simulation to produce physically plausible, simulation-ready, and intersection-free 3D scenes. Given a text prompt, PAT3D generates 3D objects, infers their spatial relations, and organizes them into a hierarchical scene tree, which is then converted into initial conditions for simulation. A differentiable rigid-body simulator ensures realistic object interactions under gravity, driving the scene toward static equilibrium without interpenetrations. To further enhance scene quality, we introduce a simulation-in-the-loop optimization procedure that guarantees physical stability and non-intersection, while improving semantic consistency with the input prompt. Experiments demonstrate that PAT3D substantially outperforms prior approaches in physical plausibility, semantic consistency, and visual quality. Beyond high-quality generation, PAT3D uniquely enables simulation-ready 3D scenes for downstream tasks such as scene editing and robotic manipulation. Code and data are available at: https://github.com/Simulation-Intelligence/PAT3D. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_21978 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | PAT3D: Physics-Augmented Text-to-3D Scene Generation Lin, Guying Huang, Kemeng Liu, Michael Gao, Ruihan Chen, Hanke Chen, Lyuhao Lu, Beijia Komura, Taku Liu, Yuan Zhu, Jun-Yan Li, Minchen Computer Vision and Pattern Recognition We introduce PAT3D, the first physics-augmented text-to-3D scene generation framework that integrates vision-language models with physics-based simulation to produce physically plausible, simulation-ready, and intersection-free 3D scenes. Given a text prompt, PAT3D generates 3D objects, infers their spatial relations, and organizes them into a hierarchical scene tree, which is then converted into initial conditions for simulation. A differentiable rigid-body simulator ensures realistic object interactions under gravity, driving the scene toward static equilibrium without interpenetrations. To further enhance scene quality, we introduce a simulation-in-the-loop optimization procedure that guarantees physical stability and non-intersection, while improving semantic consistency with the input prompt. Experiments demonstrate that PAT3D substantially outperforms prior approaches in physical plausibility, semantic consistency, and visual quality. Beyond high-quality generation, PAT3D uniquely enables simulation-ready 3D scenes for downstream tasks such as scene editing and robotic manipulation. Code and data are available at: https://github.com/Simulation-Intelligence/PAT3D. |
| title | PAT3D: Physics-Augmented Text-to-3D Scene Generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2511.21978 |