Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2508.13009 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911570821382144 |
|---|---|
| author | He, Xianglong Peng, Chunli Liu, Zexiang Wang, Boyang Zhang, Yifan Cui, Qi Kang, Fei Jiang, Biao An, Mengyin Ren, Yangyang Xu, Baixin Guo, Hao-Xiang Gong, Kaixiong Wu, Size Li, Wei Song, Xuchen Liu, Yang Li, Yangguang Zhou, Yahui |
| author_facet | He, Xianglong Peng, Chunli Liu, Zexiang Wang, Boyang Zhang, Yifan Cui, Qi Kang, Fei Jiang, Biao An, Mengyin Ren, Yangyang Xu, Baixin Guo, Hao-Xiang Gong, Kaixiong Wu, Size Li, Wei Song, Xuchen Liu, Yang Li, Yangguang Zhou, Yahui |
| contents | Recent advances in interactive video generations have demonstrated diffusion model's potential as world models by capturing complex physical dynamics and interactive behaviors. However, existing interactive world models depend on bidirectional attention and lengthy inference steps, severely limiting real-time performance. Consequently, they are hard to simulate real-world dynamics, where outcomes must update instantaneously based on historical context and current actions. To address this, we present Matrix-Game 2.0, an interactive world model generates long videos on-the-fly via few-step auto-regressive diffusion. Our framework consists of three key components: (1) A scalable data production pipeline for Unreal Engine and GTA5 environments to effectively produce massive amounts (about 1200 hours) of video data with diverse interaction annotations; (2) An action injection module that enables frame-level mouse and keyboard inputs as interactive conditions; (3) A few-step distillation based on the casual architecture for real-time and streaming video generation. Matrix Game 2.0 can generate high-quality minute-level videos across diverse scenes at an ultra-fast speed of 25 FPS. We open-source our model weights and codebase to advance research in interactive world modeling. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_13009 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Matrix-game 2.0: An open-source real-time and streaming interactive world model He, Xianglong Peng, Chunli Liu, Zexiang Wang, Boyang Zhang, Yifan Cui, Qi Kang, Fei Jiang, Biao An, Mengyin Ren, Yangyang Xu, Baixin Guo, Hao-Xiang Gong, Kaixiong Wu, Size Li, Wei Song, Xuchen Liu, Yang Li, Yangguang Zhou, Yahui Computer Vision and Pattern Recognition Recent advances in interactive video generations have demonstrated diffusion model's potential as world models by capturing complex physical dynamics and interactive behaviors. However, existing interactive world models depend on bidirectional attention and lengthy inference steps, severely limiting real-time performance. Consequently, they are hard to simulate real-world dynamics, where outcomes must update instantaneously based on historical context and current actions. To address this, we present Matrix-Game 2.0, an interactive world model generates long videos on-the-fly via few-step auto-regressive diffusion. Our framework consists of three key components: (1) A scalable data production pipeline for Unreal Engine and GTA5 environments to effectively produce massive amounts (about 1200 hours) of video data with diverse interaction annotations; (2) An action injection module that enables frame-level mouse and keyboard inputs as interactive conditions; (3) A few-step distillation based on the casual architecture for real-time and streaming video generation. Matrix Game 2.0 can generate high-quality minute-level videos across diverse scenes at an ultra-fast speed of 25 FPS. We open-source our model weights and codebase to advance research in interactive world modeling. |
| title | Matrix-game 2.0: An open-source real-time and streaming interactive world model |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2508.13009 |