Vision-Centric 4D Occupancy Forecasting and Planning via Implicit Residual World Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915783223803904 |
|---|---|
| author | Mei, Jianbiao Yang, Yu Yang, Xuemeng Wen, Licheng Lv, Jiajun Shi, Botian Liu, Yong |
| author_facet | Mei, Jianbiao Yang, Yu Yang, Xuemeng Wen, Licheng Lv, Jiajun Shi, Botian Liu, Yong |
| contents | End-to-end autonomous driving systems increasingly rely on vision-centric world models to understand and predict their environment. However, a common ineffectiveness in these models is the full reconstruction of future scenes, which expends significant capacity on redundantly modeling static backgrounds. To address this, we propose IR-WM, an Implicit Residual World Model that focuses on modeling the current state and evolution of the world. IR-WM first establishes a robust bird's-eye-view representation of the current state from the visual observation. It then leverages the BEV features from the previous timestep as a strong temporal prior and predicts only the "residual", i.e., the changes conditioned on the ego-vehicle's actions and scene context. To alleviate error accumulation over time, we further apply an alignment module to calibrate semantic and dynamic misalignments. Moreover, we investigate different forecasting-planning coupling schemes and demonstrate that the implicit future state generated by world models substantially improves planning accuracy. On the nuScenes benchmark, IR-WM achieves top performance in both 4D occupancy forecasting and trajectory planning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_16729 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Vision-Centric 4D Occupancy Forecasting and Planning via Implicit Residual World Models Mei, Jianbiao Yang, Yu Yang, Xuemeng Wen, Licheng Lv, Jiajun Shi, Botian Liu, Yong Computer Vision and Pattern Recognition End-to-end autonomous driving systems increasingly rely on vision-centric world models to understand and predict their environment. However, a common ineffectiveness in these models is the full reconstruction of future scenes, which expends significant capacity on redundantly modeling static backgrounds. To address this, we propose IR-WM, an Implicit Residual World Model that focuses on modeling the current state and evolution of the world. IR-WM first establishes a robust bird's-eye-view representation of the current state from the visual observation. It then leverages the BEV features from the previous timestep as a strong temporal prior and predicts only the "residual", i.e., the changes conditioned on the ego-vehicle's actions and scene context. To alleviate error accumulation over time, we further apply an alignment module to calibrate semantic and dynamic misalignments. Moreover, we investigate different forecasting-planning coupling schemes and demonstrate that the implicit future state generated by world models substantially improves planning accuracy. On the nuScenes benchmark, IR-WM achieves top performance in both 4D occupancy forecasting and trajectory planning. |
| title | Vision-Centric 4D Occupancy Forecasting and Planning via Implicit Residual World Models |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2510.16729 |