Martian World Model: Controllable Video Synthesis with Physically Accurate 3D Reconstructions
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915655276560384 |
|---|---|
| author | Li, Longfei Fan, Zhiwen Cong, Wenyan Liu, Xinhang Yin, Yuyang Foutter, Matt Pan, Panwang You, Chenyu Wang, Yue Wang, Zhangyang Zhao, Yao Pavone, Marco Wei, Yunchao |
| author_facet | Li, Longfei Fan, Zhiwen Cong, Wenyan Liu, Xinhang Yin, Yuyang Foutter, Matt Pan, Panwang You, Chenyu Wang, Yue Wang, Zhangyang Zhao, Yao Pavone, Marco Wei, Yunchao |
| contents | Synthesizing realistic Martian landscape videos is crucial for mission rehearsal and robotic simulation. However, this task poses unique challenges due to the scarcity of high-quality Martian data and the significant domain gap between Martian and terrestrial imagery. To address these challenges, we propose a holistic solution composed of two key components: 1) A data curation pipeline Multimodal Mars Synthesis (M3arsSynth), which reconstructs 3D Martian environments from real stereo navigation images, sourced from NASA's Planetary Data System (PDS), and renders high-fidelity multiview 3D video sequences. 2) A Martian terrain video generator, MarsGen, which synthesizes novel videos visually realistic and geometrically consistent with the 3D structure encoded in the data. Our M3arsSynth engine spans a wide range of Martian terrains and acquisition dates, enabling the generation of physically accurate 3D surface models at metric-scale resolution. MarsGen, fine-tuned on M3arsSynth data, synthesizes videos conditioned on an initial image frame and, optionally, camera trajectories or textual prompts, allowing for video generation in novel environments. Experimental results show that our approach outperforms video synthesis models trained on terrestrial datasets, achieving superior visual fidelity and 3D structural consistency. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_07978 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Martian World Model: Controllable Video Synthesis with Physically Accurate 3D Reconstructions Li, Longfei Fan, Zhiwen Cong, Wenyan Liu, Xinhang Yin, Yuyang Foutter, Matt Pan, Panwang You, Chenyu Wang, Yue Wang, Zhangyang Zhao, Yao Pavone, Marco Wei, Yunchao Computer Vision and Pattern Recognition Synthesizing realistic Martian landscape videos is crucial for mission rehearsal and robotic simulation. However, this task poses unique challenges due to the scarcity of high-quality Martian data and the significant domain gap between Martian and terrestrial imagery. To address these challenges, we propose a holistic solution composed of two key components: 1) A data curation pipeline Multimodal Mars Synthesis (M3arsSynth), which reconstructs 3D Martian environments from real stereo navigation images, sourced from NASA's Planetary Data System (PDS), and renders high-fidelity multiview 3D video sequences. 2) A Martian terrain video generator, MarsGen, which synthesizes novel videos visually realistic and geometrically consistent with the 3D structure encoded in the data. Our M3arsSynth engine spans a wide range of Martian terrains and acquisition dates, enabling the generation of physically accurate 3D surface models at metric-scale resolution. MarsGen, fine-tuned on M3arsSynth data, synthesizes videos conditioned on an initial image frame and, optionally, camera trajectories or textual prompts, allowing for video generation in novel environments. Experimental results show that our approach outperforms video synthesis models trained on terrestrial datasets, achieving superior visual fidelity and 3D structural consistency. |
| title | Martian World Model: Controllable Video Synthesis with Physically Accurate 3D Reconstructions |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2507.07978 |