Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2606.01027 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917550995013632 |
|---|---|
| author | Zhou, Pengfei Chen, Shengcong Chen, Di Wang, Jiaxu Jin, Rongjun Zhu, Bingwen Pan, Yike Gu, Songen Wang, Kuanning Nan, Shufeng Qiu, Xingyu Qiu, Chenhao Yang, Pu Cai, Yunuo Gao, Jianxiong Li, Yifan Fu, Yanwei Yue, Xiangyu Chen, Zhi Luo, Jianlan |
| author_facet | Zhou, Pengfei Chen, Shengcong Chen, Di Wang, Jiaxu Jin, Rongjun Zhu, Bingwen Pan, Yike Gu, Songen Wang, Kuanning Nan, Shufeng Qiu, Xingyu Qiu, Chenhao Yang, Pu Cai, Yunuo Gao, Jianxiong Li, Yifan Fu, Yanwei Yue, Xiangyu Chen, Zhi Luo, Jianlan |
| contents | Robotic manipulation requires models that generate executable actions while anticipating and evaluating their future consequences before physical execution. We present $τ_0$-World Model ($τ_0$-WM), a unified video-action world model that integrates policy learning, video prediction, and action evaluation within a single future-predictive framework. Built on a shared video diffusion backbone, $τ_0$-WM provides two complementary interfaces. First, a video action model jointly predicts future visual latents and continuous action chunks from multi-view observations, language instructions, and robot state. Second, an action-conditioned video simulator rolls out candidate action chunks into multi-view futures and predicts dense task-progress scores. The model is trained on approximately $27{,}300$ hours of real-robot teleoperation, UMI-style interaction, egocentric human videos, and rollout or failure trajectories using modality-specific supervision masks. At inference time, $τ_0$-WM uses test-time computation to sample action candidates, rank them with re-denoising consistency, and invoke simulator-based rectification for low-quality candidates. On challenging long-horizon and fine-grained robotic manipulation tasks, $τ_0$-WM shows superior performance over other relevant baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2606_01027 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | $τ_0$-WM: A Unified Video-Action World Model for Robotic Manipulation Zhou, Pengfei Chen, Shengcong Chen, Di Wang, Jiaxu Jin, Rongjun Zhu, Bingwen Pan, Yike Gu, Songen Wang, Kuanning Nan, Shufeng Qiu, Xingyu Qiu, Chenhao Yang, Pu Cai, Yunuo Gao, Jianxiong Li, Yifan Fu, Yanwei Yue, Xiangyu Chen, Zhi Luo, Jianlan Robotics Robotic manipulation requires models that generate executable actions while anticipating and evaluating their future consequences before physical execution. We present $τ_0$-World Model ($τ_0$-WM), a unified video-action world model that integrates policy learning, video prediction, and action evaluation within a single future-predictive framework. Built on a shared video diffusion backbone, $τ_0$-WM provides two complementary interfaces. First, a video action model jointly predicts future visual latents and continuous action chunks from multi-view observations, language instructions, and robot state. Second, an action-conditioned video simulator rolls out candidate action chunks into multi-view futures and predicts dense task-progress scores. The model is trained on approximately $27{,}300$ hours of real-robot teleoperation, UMI-style interaction, egocentric human videos, and rollout or failure trajectories using modality-specific supervision masks. At inference time, $τ_0$-WM uses test-time computation to sample action candidates, rank them with re-denoising consistency, and invoke simulator-based rectification for low-quality candidates. On challenging long-horizon and fine-grained robotic manipulation tasks, $τ_0$-WM shows superior performance over other relevant baselines. |
| title | $τ_0$-WM: A Unified Video-Action World Model for Robotic Manipulation |
| topic | Robotics |
| url | https://arxiv.org/abs/2606.01027 |