Xiaomi Auto World Model: A Joint World Model Integrating Reconstruction and Generation for Autonomous Driving
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913168022831104 |
|---|---|
| author | Zhou, Lijun Luo, Hongcheng Zhu, Zhenxin Chi, Cheng Tu, Mingfei Xiong, Kaixin Gong, Lei Wu, Zhanqian Zhang, Zehan Li, Fangzhen Li, Hao Shen, Yingying He, Jiale Zhu, Haohui Zhao, Shan Wang, Kai Zhan, Zhiwei Pu, Yuechuan Tan, Kaiyuan Yang, Ruiling Wang, Xianqi Yan, Tianyi Zhou, Jiawei Zhang, Lei Zhao, Jingyang Zhou, Xi Sun, Chitian Wu, Chenming Deng, Jiong Xie, Hongwei Lu, Ming Ma, Kun Chen, Long Chen, Guang Ye, Hangjun Wang, Bing Sun, Haiyang |
| author_facet | Zhou, Lijun Luo, Hongcheng Zhu, Zhenxin Chi, Cheng Tu, Mingfei Xiong, Kaixin Gong, Lei Wu, Zhanqian Zhang, Zehan Li, Fangzhen Li, Hao Shen, Yingying He, Jiale Zhu, Haohui Zhao, Shan Wang, Kai Zhan, Zhiwei Pu, Yuechuan Tan, Kaiyuan Yang, Ruiling Wang, Xianqi Yan, Tianyi Zhou, Jiawei Zhang, Lei Zhao, Jingyang Zhou, Xi Sun, Chitian Wu, Chenming Deng, Jiong Xie, Hongwei Lu, Ming Ma, Kun Chen, Long Chen, Guang Ye, Hangjun Wang, Bing Sun, Haiyang |
| contents | This report presents a unified technical system addressing the two core capabilities of world models for autonomous driving: world representation and world generation. For world representation, we propose WorldRec, a feed-forward reconstruction architecture driven by sparse scene queries. WorldRec initializes structured queries in 3D space, leveraging them to aggregate cross-view, cross-temporal features, thereby naturally enforcing spatial consistency across frames and yielding compact yet high-fidelity 3D Gaussian scene representations. For world generation, we propose WorldGen, a two-stage training framework of bidirectional pretraining followed by causal fine-tuning through three progressive stages (Teacher Forcing, ODE distillation, and DMD), enabling high-quality online causal video generation in as few as 4 denoising steps. Building on both modules, we further introduce the JWM, which deeply integrates WorldRec and WorldGen to achieve synergistic gains in generation stability, cross-frame consistency, and visual fidelity, providing a solid foundation for closed-loop simulation, data synthesis, and end-to-end training in autonomous driving. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_18137 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Xiaomi Auto World Model: A Joint World Model Integrating Reconstruction and Generation for Autonomous Driving Zhou, Lijun Luo, Hongcheng Zhu, Zhenxin Chi, Cheng Tu, Mingfei Xiong, Kaixin Gong, Lei Wu, Zhanqian Zhang, Zehan Li, Fangzhen Li, Hao Shen, Yingying He, Jiale Zhu, Haohui Zhao, Shan Wang, Kai Zhan, Zhiwei Pu, Yuechuan Tan, Kaiyuan Yang, Ruiling Wang, Xianqi Yan, Tianyi Zhou, Jiawei Zhang, Lei Zhao, Jingyang Zhou, Xi Sun, Chitian Wu, Chenming Deng, Jiong Xie, Hongwei Lu, Ming Ma, Kun Chen, Long Chen, Guang Ye, Hangjun Wang, Bing Sun, Haiyang Computer Vision and Pattern Recognition This report presents a unified technical system addressing the two core capabilities of world models for autonomous driving: world representation and world generation. For world representation, we propose WorldRec, a feed-forward reconstruction architecture driven by sparse scene queries. WorldRec initializes structured queries in 3D space, leveraging them to aggregate cross-view, cross-temporal features, thereby naturally enforcing spatial consistency across frames and yielding compact yet high-fidelity 3D Gaussian scene representations. For world generation, we propose WorldGen, a two-stage training framework of bidirectional pretraining followed by causal fine-tuning through three progressive stages (Teacher Forcing, ODE distillation, and DMD), enabling high-quality online causal video generation in as few as 4 denoising steps. Building on both modules, we further introduce the JWM, which deeply integrates WorldRec and WorldGen to achieve synergistic gains in generation stability, cross-frame consistency, and visual fidelity, providing a solid foundation for closed-loop simulation, data synthesis, and end-to-end training in autonomous driving. |
| title | Xiaomi Auto World Model: A Joint World Model Integrating Reconstruction and Generation for Autonomous Driving |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2605.18137 |