TeleWorld: Towards Dynamic Multimodal Synthesis with a 4D World Model
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917179785478144 |
|---|---|
| author | Chen, Yabo Liang, Yuanzhi Wang, Jiepeng Chen, Tingxi Cheng, Junfei Gu, Zixiao Huang, Yuyang Jiang, Zicheng Li, Wei Li, Tian Li, Weichen Li, Zuoxin Liu, Guangce Liu, Jialun Liu, Junqi Wang, Haoyuan Weng, Qizhen Wu, Xuan'er Xiang, Xunzhi Yang, Xiaoyan Zhang, Xin Zhang, Shiwen Zhou, Junyu Zhou, Chengcheng Huang, Haibin Zhang, Chi Li, Xuelong |
| author_facet | Chen, Yabo Liang, Yuanzhi Wang, Jiepeng Chen, Tingxi Cheng, Junfei Gu, Zixiao Huang, Yuyang Jiang, Zicheng Li, Wei Li, Tian Li, Weichen Li, Zuoxin Liu, Guangce Liu, Jialun Liu, Junqi Wang, Haoyuan Weng, Qizhen Wu, Xuan'er Xiang, Xunzhi Yang, Xiaoyan Zhang, Xin Zhang, Shiwen Zhou, Junyu Zhou, Chengcheng Huang, Haibin Zhang, Chi Li, Xuelong |
| contents | World models aim to endow AI systems with the ability to represent, generate, and interact with dynamic environments in a coherent and temporally consistent manner. While recent video generation models have demonstrated impressive visual quality, they remain limited in real-time interaction, long-horizon consistency, and persistent memory of dynamic scenes, hindering their evolution into practical world models. In this report, we present TeleWorld, a real-time multimodal 4D world modeling framework that unifies video generation, dynamic scene reconstruction, and long-term world memory within a closed-loop system. TeleWorld introduces a novel generation-reconstruction-guidance paradigm, where generated video streams are continuously reconstructed into a dynamic 4D spatio-temporal representation, which in turn guides subsequent generation to maintain spatial, temporal, and physical consistency. To support long-horizon generation with low latency, we employ an autoregressive diffusion-based video model enhanced with Macro-from-Micro Planning (MMPL)--a hierarchical planning method that reduces error accumulation from frame-level to segment-level-alongside efficient Distribution Matching Distillation (DMD), enabling real-time synthesis under practical computational budgets. Our approach achieves seamless integration of dynamic object modeling and static scene representation within a unified 4D framework, advancing world models toward practical, interactive, and computationally accessible systems. Extensive experiments demonstrate that TeleWorld achieves strong performance in both static and dynamic world understanding, long-term consistency, and real-time generation efficiency, positioning it as a practical step toward interactive, memory-enabled world models for multimodal generation and embodied intelligence. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_00051 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | TeleWorld: Towards Dynamic Multimodal Synthesis with a 4D World Model Chen, Yabo Liang, Yuanzhi Wang, Jiepeng Chen, Tingxi Cheng, Junfei Gu, Zixiao Huang, Yuyang Jiang, Zicheng Li, Wei Li, Tian Li, Weichen Li, Zuoxin Liu, Guangce Liu, Jialun Liu, Junqi Wang, Haoyuan Weng, Qizhen Wu, Xuan'er Xiang, Xunzhi Yang, Xiaoyan Zhang, Xin Zhang, Shiwen Zhou, Junyu Zhou, Chengcheng Huang, Haibin Zhang, Chi Li, Xuelong Computer Vision and Pattern Recognition World models aim to endow AI systems with the ability to represent, generate, and interact with dynamic environments in a coherent and temporally consistent manner. While recent video generation models have demonstrated impressive visual quality, they remain limited in real-time interaction, long-horizon consistency, and persistent memory of dynamic scenes, hindering their evolution into practical world models. In this report, we present TeleWorld, a real-time multimodal 4D world modeling framework that unifies video generation, dynamic scene reconstruction, and long-term world memory within a closed-loop system. TeleWorld introduces a novel generation-reconstruction-guidance paradigm, where generated video streams are continuously reconstructed into a dynamic 4D spatio-temporal representation, which in turn guides subsequent generation to maintain spatial, temporal, and physical consistency. To support long-horizon generation with low latency, we employ an autoregressive diffusion-based video model enhanced with Macro-from-Micro Planning (MMPL)--a hierarchical planning method that reduces error accumulation from frame-level to segment-level-alongside efficient Distribution Matching Distillation (DMD), enabling real-time synthesis under practical computational budgets. Our approach achieves seamless integration of dynamic object modeling and static scene representation within a unified 4D framework, advancing world models toward practical, interactive, and computationally accessible systems. Extensive experiments demonstrate that TeleWorld achieves strong performance in both static and dynamic world understanding, long-term consistency, and real-time generation efficiency, positioning it as a practical step toward interactive, memory-enabled world models for multimodal generation and embodied intelligence. |
| title | TeleWorld: Towards Dynamic Multimodal Synthesis with a 4D World Model |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2601.00051 |