OmniNWM: Omniscient Driving Navigation World Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911267554328576 |
|---|---|
| author | Li, Bohan Ma, Zhuang Du, Dalong Peng, Baorui Liang, Zhujin Liu, Zhenqiang Ma, Chao Jin, Yueming Zhao, Hao Zeng, Wenjun Jin, Xin |
| author_facet | Li, Bohan Ma, Zhuang Du, Dalong Peng, Baorui Liang, Zhujin Liu, Zhenqiang Ma, Chao Jin, Yueming Zhao, Hao Zeng, Wenjun Jin, Xin |
| contents | Autonomous driving world models are expected to work effectively across three core dimensions: state, action, and reward. Existing models, however, are typically restricted to limited state modalities, short video sequences, imprecise action control, and a lack of reward awareness. In this paper, we introduce OmniNWM, an omniscient panoramic navigation world model that addresses all three dimensions within a unified framework. For state, OmniNWM jointly generates panoramic videos of RGB, semantics, metric depth, and 3D occupancy. A flexible forcing strategy enables high-quality long-horizon auto-regressive generation. For action, we introduce a normalized panoramic Plucker ray-map representation that encodes input trajectories into pixel-level signals, enabling highly precise and generalizable control over panoramic video generation. Regarding reward, we move beyond learning reward functions with external image-based models: instead, we leverage the generated 3D occupancy to directly define rule-based dense rewards for driving compliance and safety. Extensive experiments demonstrate that OmniNWM achieves state-of-the-art performance in video generation, control accuracy, and long-horizon stability, while providing a reliable closed-loop evaluation framework through occupancy-grounded rewards. Project page is available at https://arlo0o.github.io/OmniNWM/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_18313 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | OmniNWM: Omniscient Driving Navigation World Models Li, Bohan Ma, Zhuang Du, Dalong Peng, Baorui Liang, Zhujin Liu, Zhenqiang Ma, Chao Jin, Yueming Zhao, Hao Zeng, Wenjun Jin, Xin Computer Vision and Pattern Recognition Autonomous driving world models are expected to work effectively across three core dimensions: state, action, and reward. Existing models, however, are typically restricted to limited state modalities, short video sequences, imprecise action control, and a lack of reward awareness. In this paper, we introduce OmniNWM, an omniscient panoramic navigation world model that addresses all three dimensions within a unified framework. For state, OmniNWM jointly generates panoramic videos of RGB, semantics, metric depth, and 3D occupancy. A flexible forcing strategy enables high-quality long-horizon auto-regressive generation. For action, we introduce a normalized panoramic Plucker ray-map representation that encodes input trajectories into pixel-level signals, enabling highly precise and generalizable control over panoramic video generation. Regarding reward, we move beyond learning reward functions with external image-based models: instead, we leverage the generated 3D occupancy to directly define rule-based dense rewards for driving compliance and safety. Extensive experiments demonstrate that OmniNWM achieves state-of-the-art performance in video generation, control accuracy, and long-horizon stability, while providing a reliable closed-loop evaluation framework through occupancy-grounded rewards. Project page is available at https://arlo0o.github.io/OmniNWM/. |
| title | OmniNWM: Omniscient Driving Navigation World Models |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2510.18313 |