Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2510.18135 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909860735483904 |
|---|---|
| author | Zhang, Jiahan Jiang, Muqing Dai, Nanru Lu, Taiming Uzunoglu, Arda Zhang, Shunchi Wei, Yana Wang, Jiahao Patel, Vishal M. Liang, Paul Pu Khashabi, Daniel Peng, Cheng Chellappa, Rama Shu, Tianmin Yuille, Alan Du, Yilun Chen, Jieneng |
| author_facet | Zhang, Jiahan Jiang, Muqing Dai, Nanru Lu, Taiming Uzunoglu, Arda Zhang, Shunchi Wei, Yana Wang, Jiahao Patel, Vishal M. Liang, Paul Pu Khashabi, Daniel Peng, Cheng Chellappa, Rama Shu, Tianmin Yuille, Alan Du, Yilun Chen, Jieneng |
| contents | Generative world models (WMs) can now simulate worlds with striking visual realism, which naturally raises the question of whether they can endow embodied agents with predictive perception for decision making. Progress on this question has been limited by fragmented evaluation: most existing benchmarks adopt open-loop protocols that emphasize visual quality in isolation, leaving the core issue of embodied utility unresolved, i.e., do WMs actually help agents succeed at embodied tasks? To address this gap, we introduce World-in-World, the first open platform that benchmarks WMs in a closed-loop world that mirrors real agent-environment interactions. World-in-World provides a unified online planning strategy and a standardized action API, enabling heterogeneous WMs for decision making. We curate four closed-loop environments that rigorously evaluate diverse WMs, prioritize task success as the primary metric, and move beyond the common focus on visual quality; we also present the first data scaling law for world models in embodied settings. Our study uncovers three surprises: (1) visual quality alone does not guarantee task success, controllability matters more; (2) scaling post-training with action-observation data is more effective than upgrading the pretrained video generators; and (3) allocating more inference-time compute allows WMs to substantially improve closed-loop performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_18135 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | World-in-World: World Models in a Closed-Loop World Zhang, Jiahan Jiang, Muqing Dai, Nanru Lu, Taiming Uzunoglu, Arda Zhang, Shunchi Wei, Yana Wang, Jiahao Patel, Vishal M. Liang, Paul Pu Khashabi, Daniel Peng, Cheng Chellappa, Rama Shu, Tianmin Yuille, Alan Du, Yilun Chen, Jieneng Computer Vision and Pattern Recognition Generative world models (WMs) can now simulate worlds with striking visual realism, which naturally raises the question of whether they can endow embodied agents with predictive perception for decision making. Progress on this question has been limited by fragmented evaluation: most existing benchmarks adopt open-loop protocols that emphasize visual quality in isolation, leaving the core issue of embodied utility unresolved, i.e., do WMs actually help agents succeed at embodied tasks? To address this gap, we introduce World-in-World, the first open platform that benchmarks WMs in a closed-loop world that mirrors real agent-environment interactions. World-in-World provides a unified online planning strategy and a standardized action API, enabling heterogeneous WMs for decision making. We curate four closed-loop environments that rigorously evaluate diverse WMs, prioritize task success as the primary metric, and move beyond the common focus on visual quality; we also present the first data scaling law for world models in embodied settings. Our study uncovers three surprises: (1) visual quality alone does not guarantee task success, controllability matters more; (2) scaling post-training with action-observation data is more effective than upgrading the pretrained video generators; and (3) allocating more inference-time compute allows WMs to substantially improve closed-loop performance. |
| title | World-in-World: World Models in a Closed-Loop World |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2510.18135 |