Motus: A Unified Latent Action World Model
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917169270358016 |
|---|---|
| author | Bi, Hongzhe Tan, Hengkai Xie, Shenghao Wang, Zeyuan Huang, Shuhe Liu, Haitian Zhao, Ruowen Feng, Yao Xiang, Chendong Rong, Yinze Zhao, Hongyan Liu, Hanyu Su, Zhizhong Ma, Lei Su, Hang Zhu, Jun |
| author_facet | Bi, Hongzhe Tan, Hengkai Xie, Shenghao Wang, Zeyuan Huang, Shuhe Liu, Haitian Zhao, Ruowen Feng, Yao Xiang, Chendong Rong, Yinze Zhao, Hongyan Liu, Hanyu Su, Zhizhong Ma, Lei Su, Hang Zhu, Jun |
| contents | While a general embodied agent must function as a unified system, current methods are built on isolated models for understanding, world modeling, and control. This fragmentation prevents unifying multimodal generative capabilities and hinders learning from large-scale, heterogeneous data. In this paper, we propose Motus, a unified latent action world model that leverages existing general pretrained models and rich, sharable motion information. Motus introduces a Mixture-of-Transformer (MoT) architecture to integrate three experts (i.e., understanding, video generation, and action) and adopts a UniDiffuser-style scheduler to enable flexible switching between different modeling modes (i.e., world models, vision-language-action models, inverse dynamics models, video generation models, and video-action joint prediction models). Motus further leverages the optical flow to learn latent actions and adopts a recipe with three-phase training pipeline and six-layer data pyramid, thereby extracting pixel-level "delta action" and enabling large-scale action pretraining. Experiments show that Motus achieves superior performance against state-of-the-art methods in both simulation (a +15% improvement over X-VLA and a +45% improvement over Pi0.5) and real-world scenarios(improved by +11~48%), demonstrating unified modeling of all functionalities and priors significantly benefits downstream robotic tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_13030 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Motus: A Unified Latent Action World Model Bi, Hongzhe Tan, Hengkai Xie, Shenghao Wang, Zeyuan Huang, Shuhe Liu, Haitian Zhao, Ruowen Feng, Yao Xiang, Chendong Rong, Yinze Zhao, Hongyan Liu, Hanyu Su, Zhizhong Ma, Lei Su, Hang Zhu, Jun Computer Vision and Pattern Recognition Machine Learning Robotics While a general embodied agent must function as a unified system, current methods are built on isolated models for understanding, world modeling, and control. This fragmentation prevents unifying multimodal generative capabilities and hinders learning from large-scale, heterogeneous data. In this paper, we propose Motus, a unified latent action world model that leverages existing general pretrained models and rich, sharable motion information. Motus introduces a Mixture-of-Transformer (MoT) architecture to integrate three experts (i.e., understanding, video generation, and action) and adopts a UniDiffuser-style scheduler to enable flexible switching between different modeling modes (i.e., world models, vision-language-action models, inverse dynamics models, video generation models, and video-action joint prediction models). Motus further leverages the optical flow to learn latent actions and adopts a recipe with three-phase training pipeline and six-layer data pyramid, thereby extracting pixel-level "delta action" and enabling large-scale action pretraining. Experiments show that Motus achieves superior performance against state-of-the-art methods in both simulation (a +15% improvement over X-VLA and a +45% improvement over Pi0.5) and real-world scenarios(improved by +11~48%), demonstrating unified modeling of all functionalities and priors significantly benefits downstream robotic tasks. |
| title | Motus: A Unified Latent Action World Model |
| topic | Computer Vision and Pattern Recognition Machine Learning Robotics |
| url | https://arxiv.org/abs/2512.13030 |