OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modeling
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916966566985728 |
|---|---|
| author | Zhou, Yang Wang, Yifan Zhou, Jianjun Chang, Wenzheng Guo, Haoyu Li, Zizun Ma, Kaijing Li, Xinyue Wang, Yating Zhu, Haoyi Liu, Mingyu Liu, Dingning Yang, Jiange Fu, Zhoujie Chen, Junyi Shen, Chunhua Pang, Jiangmiao Zhang, Kaipeng He, Tong |
| author_facet | Zhou, Yang Wang, Yifan Zhou, Jianjun Chang, Wenzheng Guo, Haoyu Li, Zizun Ma, Kaijing Li, Xinyue Wang, Yating Zhu, Haoyi Liu, Mingyu Liu, Dingning Yang, Jiange Fu, Zhoujie Chen, Junyi Shen, Chunhua Pang, Jiangmiao Zhang, Kaipeng He, Tong |
| contents | The field of 4D world modeling - aiming to jointly capture spatial geometry and temporal dynamics - has witnessed remarkable progress in recent years, driven by advances in large-scale generative models and multimodal learning. However, the development of truly general 4D world models remains fundamentally constrained by the availability of high-quality data. Existing datasets and benchmarks often lack the dynamic complexity, multi-domain diversity, and spatial-temporal annotations required to support key tasks such as 4D geometric reconstruction, future prediction, and camera-control video generation. To address this gap, we introduce OmniWorld, a large-scale, multi-domain, multi-modal dataset specifically designed for 4D world modeling. OmniWorld consists of a newly collected OmniWorld-Game dataset and several curated public datasets spanning diverse domains. Compared with existing synthetic datasets, OmniWorld-Game provides richer modality coverage, larger scale, and more realistic dynamic interactions. Based on this dataset, we establish a challenging benchmark that exposes the limitations of current state-of-the-art (SOTA) approaches in modeling complex 4D environments. Moreover, fine-tuning existing SOTA methods on OmniWorld leads to significant performance gains across 4D reconstruction and video generation tasks, strongly validating OmniWorld as a powerful resource for training and evaluation. We envision OmniWorld as a catalyst for accelerating the development of general-purpose 4D world models, ultimately advancing machines' holistic understanding of the physical world. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_12201 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modeling Zhou, Yang Wang, Yifan Zhou, Jianjun Chang, Wenzheng Guo, Haoyu Li, Zizun Ma, Kaijing Li, Xinyue Wang, Yating Zhu, Haoyi Liu, Mingyu Liu, Dingning Yang, Jiange Fu, Zhoujie Chen, Junyi Shen, Chunhua Pang, Jiangmiao Zhang, Kaipeng He, Tong Computer Vision and Pattern Recognition The field of 4D world modeling - aiming to jointly capture spatial geometry and temporal dynamics - has witnessed remarkable progress in recent years, driven by advances in large-scale generative models and multimodal learning. However, the development of truly general 4D world models remains fundamentally constrained by the availability of high-quality data. Existing datasets and benchmarks often lack the dynamic complexity, multi-domain diversity, and spatial-temporal annotations required to support key tasks such as 4D geometric reconstruction, future prediction, and camera-control video generation. To address this gap, we introduce OmniWorld, a large-scale, multi-domain, multi-modal dataset specifically designed for 4D world modeling. OmniWorld consists of a newly collected OmniWorld-Game dataset and several curated public datasets spanning diverse domains. Compared with existing synthetic datasets, OmniWorld-Game provides richer modality coverage, larger scale, and more realistic dynamic interactions. Based on this dataset, we establish a challenging benchmark that exposes the limitations of current state-of-the-art (SOTA) approaches in modeling complex 4D environments. Moreover, fine-tuning existing SOTA methods on OmniWorld leads to significant performance gains across 4D reconstruction and video generation tasks, strongly validating OmniWorld as a powerful resource for training and evaluation. We envision OmniWorld as a catalyst for accelerating the development of general-purpose 4D world models, ultimately advancing machines' holistic understanding of the physical world. |
| title | OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modeling |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2509.12201 |