Mirage2Matter: A Physically Grounded Gaussian World Model from Video
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910006565142528 |
|---|---|
| author | Gao, Zhengqing Li, Ziwen Wang, Xin Huang, Jiaxin Ren, Zhenyang Shao, Mingkai Zhang, Hanlue Huang, Tianyu Cheng, Yongkang Guo, Yandong Lin, Runqi Wang, Yuanyuan Liu, Tongliang Zhang, Kun Gong, Mingming |
| author_facet | Gao, Zhengqing Li, Ziwen Wang, Xin Huang, Jiaxin Ren, Zhenyang Shao, Mingkai Zhang, Hanlue Huang, Tianyu Cheng, Yongkang Guo, Yandong Lin, Runqi Wang, Yuanyuan Liu, Tongliang Zhang, Kun Gong, Mingming |
| contents | The scalability of embodied intelligence is fundamentally constrained by the scarcity of real-world interaction data. While simulation platforms provide a promising alternative, existing approaches often suffer from a substantial visual and physical gap to real environments and rely on expensive sensors, precise robot calibration, or depth measurements, limiting their practicality at scale. We present Simulate Anything, a graphics-driven world modeling and simulation framework that enables efficient generation of high-fidelity embodied training data using only multi-view environment videos and off-the-shelf assets. Our approach reconstructs real-world environments into a photorealistic scene representation using 3D Gaussian Splatting (3DGS), seamlessly capturing fine-grained geometry and appearance from video. We then leverage generative models to recover a physically realistic representation and integrate it into a simulation environment via a precision calibration target, enabling accurate scale alignment between the reconstructed scene and the real world. Together, these components provide a unified, editable, and physically grounded world model. Vision Language Action (VLA) models trained on our simulated data achieve strong zero-shot performance on downstream tasks, matching or even surpassing results obtained with real-world data, highlighting the potential of reconstruction-driven world modeling for scalable and practical embodied intelligence training. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_00096 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Mirage2Matter: A Physically Grounded Gaussian World Model from Video Gao, Zhengqing Li, Ziwen Wang, Xin Huang, Jiaxin Ren, Zhenyang Shao, Mingkai Zhang, Hanlue Huang, Tianyu Cheng, Yongkang Guo, Yandong Lin, Runqi Wang, Yuanyuan Liu, Tongliang Zhang, Kun Gong, Mingming Computer Vision and Pattern Recognition Artificial Intelligence The scalability of embodied intelligence is fundamentally constrained by the scarcity of real-world interaction data. While simulation platforms provide a promising alternative, existing approaches often suffer from a substantial visual and physical gap to real environments and rely on expensive sensors, precise robot calibration, or depth measurements, limiting their practicality at scale. We present Simulate Anything, a graphics-driven world modeling and simulation framework that enables efficient generation of high-fidelity embodied training data using only multi-view environment videos and off-the-shelf assets. Our approach reconstructs real-world environments into a photorealistic scene representation using 3D Gaussian Splatting (3DGS), seamlessly capturing fine-grained geometry and appearance from video. We then leverage generative models to recover a physically realistic representation and integrate it into a simulation environment via a precision calibration target, enabling accurate scale alignment between the reconstructed scene and the real world. Together, these components provide a unified, editable, and physically grounded world model. Vision Language Action (VLA) models trained on our simulated data achieve strong zero-shot performance on downstream tasks, matching or even surpassing results obtained with real-world data, highlighting the potential of reconstruction-driven world modeling for scalable and practical embodied intelligence training. |
| title | Mirage2Matter: A Physically Grounded Gaussian World Model from Video |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2602.00096 |