ReWorld: Multi-Dimensional Reward Modeling for Embodied World Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912831682641920 |
|---|---|
| author | Peng, Baorui Zhang, Wenyao Xu, Liang Qi, Zekun Zhang, Jiazhao Liu, Hongsi Zeng, Wenjun Jin, Xin |
| author_facet | Peng, Baorui Zhang, Wenyao Xu, Liang Qi, Zekun Zhang, Jiazhao Liu, Hongsi Zeng, Wenjun Jin, Xin |
| contents | Recently, video-based world models that learn to simulate the dynamics have gained increasing attention in robot learning. However, current approaches primarily emphasize visual generative quality while overlooking physical fidelity, dynamic consistency, and task logic, especially for contact-rich manipulation tasks, which limits their applicability to downstream tasks. To this end, we introduce ReWorld, a framework aimed to employ reinforcement learning to align the video-based embodied world models with physical realism, task completion capability, embodiment plausibility and visual quality. Specifically, we first construct a large-scale (~235K) video preference dataset and employ it to train a hierarchical reward model designed to capture multi-dimensional reward consistent with human preferences. We further propose a practical alignment algorithm that post-trains flow-based world models using this reward through a computationally efficient PPO-style algorithm. Comprehensive experiments and theoretical analysis demonstrate that ReWorld significantly improves the physical fidelity, logical coherence, embodiment and visual quality of generated rollouts, outperforming previous methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_12428 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | ReWorld: Multi-Dimensional Reward Modeling for Embodied World Models Peng, Baorui Zhang, Wenyao Xu, Liang Qi, Zekun Zhang, Jiazhao Liu, Hongsi Zeng, Wenjun Jin, Xin Robotics Computer Vision and Pattern Recognition Recently, video-based world models that learn to simulate the dynamics have gained increasing attention in robot learning. However, current approaches primarily emphasize visual generative quality while overlooking physical fidelity, dynamic consistency, and task logic, especially for contact-rich manipulation tasks, which limits their applicability to downstream tasks. To this end, we introduce ReWorld, a framework aimed to employ reinforcement learning to align the video-based embodied world models with physical realism, task completion capability, embodiment plausibility and visual quality. Specifically, we first construct a large-scale (~235K) video preference dataset and employ it to train a hierarchical reward model designed to capture multi-dimensional reward consistent with human preferences. We further propose a practical alignment algorithm that post-trains flow-based world models using this reward through a computationally efficient PPO-style algorithm. Comprehensive experiments and theoretical analysis demonstrate that ReWorld significantly improves the physical fidelity, logical coherence, embodiment and visual quality of generated rollouts, outperforming previous methods. |
| title | ReWorld: Multi-Dimensional Reward Modeling for Embodied World Models |
| topic | Robotics Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2601.12428 |