Towards Unified World Models for Visual Navigation via Memory-Augmented Planning and Foresight
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917355899060224 |
|---|---|
| author | Dong, Yifei Wu, Fengyi Chen, Guangyu Kong, Lingdong Zhu, Xu Hu, Qiyu Zhou, Yuxuan Sun, Jingdong He, Jun-Yan Dai, Qi Hauptmann, Alexander G. Cheng, Zhi-Qi |
| author_facet | Dong, Yifei Wu, Fengyi Chen, Guangyu Kong, Lingdong Zhu, Xu Hu, Qiyu Zhou, Yuxuan Sun, Jingdong He, Jun-Yan Dai, Qi Hauptmann, Alexander G. Cheng, Zhi-Qi |
| contents | Enabling embodied agents to imagine future states is essential for robust and generalizable visual navigation. Yet, state-of-the-art systems typically rely on modular designs that decouple navigation planning from visual world modeling, which often induces state-action misalignment and weak adaptability in novel or dynamic scenarios. We propose UniWM, a unified, memory-augmented world model that integrates egocentric visual foresight and planning within a single multimodal autoregressive backbone. UniWM explicitly grounds action selection in visually imagined outcomes, tightly aligning prediction with control. Meanwhile, a hierarchical memory mechanism fuses short-term perceptual cues with longer-term trajectory context, supporting stable and coherent reasoning over extended horizons. Extensive experiments on four challenging benchmarks (Go Stanford, ReCon, SCAND, HuRoN) and the 1X Humanoid Dataset show that UniWM improves navigation success rates by up to 30%, substantially reduces trajectory errors against strong baselines, generalizes zero-shot to the unseen TartanDrive dataset, and scales naturally to high-dimensional humanoid control. These results position UniWM as a principled step toward unified, imagination-driven embodied navigation. The code and models are available at https://github.com/F1y1113/UniWM. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_08713 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards Unified World Models for Visual Navigation via Memory-Augmented Planning and Foresight Dong, Yifei Wu, Fengyi Chen, Guangyu Kong, Lingdong Zhu, Xu Hu, Qiyu Zhou, Yuxuan Sun, Jingdong He, Jun-Yan Dai, Qi Hauptmann, Alexander G. Cheng, Zhi-Qi Artificial Intelligence Computer Vision and Pattern Recognition Robotics Enabling embodied agents to imagine future states is essential for robust and generalizable visual navigation. Yet, state-of-the-art systems typically rely on modular designs that decouple navigation planning from visual world modeling, which often induces state-action misalignment and weak adaptability in novel or dynamic scenarios. We propose UniWM, a unified, memory-augmented world model that integrates egocentric visual foresight and planning within a single multimodal autoregressive backbone. UniWM explicitly grounds action selection in visually imagined outcomes, tightly aligning prediction with control. Meanwhile, a hierarchical memory mechanism fuses short-term perceptual cues with longer-term trajectory context, supporting stable and coherent reasoning over extended horizons. Extensive experiments on four challenging benchmarks (Go Stanford, ReCon, SCAND, HuRoN) and the 1X Humanoid Dataset show that UniWM improves navigation success rates by up to 30%, substantially reduces trajectory errors against strong baselines, generalizes zero-shot to the unseen TartanDrive dataset, and scales naturally to high-dimensional humanoid control. These results position UniWM as a principled step toward unified, imagination-driven embodied navigation. The code and models are available at https://github.com/F1y1113/UniWM. |
| title | Towards Unified World Models for Visual Navigation via Memory-Augmented Planning and Foresight |
| topic | Artificial Intelligence Computer Vision and Pattern Recognition Robotics |
| url | https://arxiv.org/abs/2510.08713 |