Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2511.22039 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910127039184896 |
|---|---|
| author | Du, Jiayuan Zhao, Yiming Guo, Zhenglong Pan, Yong Hou, Wenbo Hao, Zhihui Zhan, Kun Chen, Qijun |
| author_facet | Du, Jiayuan Zhao, Yiming Guo, Zhenglong Pan, Yong Hou, Wenbo Hao, Zhihui Zhan, Kun Chen, Qijun |
| contents | This paper introduces a novel architecture for trajectory-conditioned forecasting of future 3D scene occupancy. In contrast to methods that rely on variational autoencoders (VAEs) to generate discrete occupancy tokens, which inherently limit representational capacity, our approach predicts multi-frame future occupancy in an end-to-end manner directly from raw image features. Inspired by the success of attention-based transformer architectures in foundational vision and language models such as GPT and VGGT, we employ a sparse occupancy representation that bypasses the intermediate bird's eye view (BEV) projection and its explicit geometric priors. This design allows the transformer to capture spatiotemporal dependencies more effectively. By avoiding both the finite-capacity constraint of discrete tokenization and the structural limitations of BEV representations, our method achieves state-of-the-art performance on the nuScenes benchmark for 1-3 second occupancy forecasting, outperforming existing approaches by a significant margin. Furthermore, it demonstrates robust scene dynamics understanding, consistently delivering high accuracy under arbitrary future trajectory conditioning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_22039 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SparseWorld-TC: Trajectory-Conditioned Sparse Occupancy World Model Du, Jiayuan Zhao, Yiming Guo, Zhenglong Pan, Yong Hou, Wenbo Hao, Zhihui Zhan, Kun Chen, Qijun Computer Vision and Pattern Recognition This paper introduces a novel architecture for trajectory-conditioned forecasting of future 3D scene occupancy. In contrast to methods that rely on variational autoencoders (VAEs) to generate discrete occupancy tokens, which inherently limit representational capacity, our approach predicts multi-frame future occupancy in an end-to-end manner directly from raw image features. Inspired by the success of attention-based transformer architectures in foundational vision and language models such as GPT and VGGT, we employ a sparse occupancy representation that bypasses the intermediate bird's eye view (BEV) projection and its explicit geometric priors. This design allows the transformer to capture spatiotemporal dependencies more effectively. By avoiding both the finite-capacity constraint of discrete tokenization and the structural limitations of BEV representations, our method achieves state-of-the-art performance on the nuScenes benchmark for 1-3 second occupancy forecasting, outperforming existing approaches by a significant margin. Furthermore, it demonstrates robust scene dynamics understanding, consistently delivering high accuracy under arbitrary future trajectory conditioning. |
| title | SparseWorld-TC: Trajectory-Conditioned Sparse Occupancy World Model |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2511.22039 |