UniScene: Unified Occupancy-centric Driving Scene Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913729676836864 |
|---|---|
| author | Li, Bohan Guo, Jiazhe Liu, Hongsi Zou, Yingshuang Ding, Yikang Chen, Xiwu Zhu, Hu Tan, Feiyang Zhang, Chi Wang, Tiancai Zhou, Shuchang Zhang, Li Qi, Xiaojuan Zhao, Hao Yang, Mu Zeng, Wenjun Jin, Xin |
| author_facet | Li, Bohan Guo, Jiazhe Liu, Hongsi Zou, Yingshuang Ding, Yikang Chen, Xiwu Zhu, Hu Tan, Feiyang Zhang, Chi Wang, Tiancai Zhou, Shuchang Zhang, Li Qi, Xiaojuan Zhao, Hao Yang, Mu Zeng, Wenjun Jin, Xin |
| contents | Generating high-fidelity, controllable, and annotated training data is critical for autonomous driving. Existing methods typically generate a single data form directly from a coarse scene layout, which not only fails to output rich data forms required for diverse downstream tasks but also struggles to model the direct layout-to-data distribution. In this paper, we introduce UniScene, the first unified framework for generating three key data forms - semantic occupancy, video, and LiDAR - in driving scenes. UniScene employs a progressive generation process that decomposes the complex task of scene generation into two hierarchical steps: (a) first generating semantic occupancy from a customized scene layout as a meta scene representation rich in both semantic and geometric information, and then (b) conditioned on occupancy, generating video and LiDAR data, respectively, with two novel transfer strategies of Gaussian-based Joint Rendering and Prior-guided Sparse Modeling. This occupancy-centric approach reduces the generation burden, especially for intricate scenes, while providing detailed intermediate representations for the subsequent generation stages. Extensive experiments demonstrate that UniScene outperforms previous SOTAs in the occupancy, video, and LiDAR generation, which also indeed benefits downstream driving tasks. Project page: https://arlo0o.github.io/uniscene/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_05435 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | UniScene: Unified Occupancy-centric Driving Scene Generation Li, Bohan Guo, Jiazhe Liu, Hongsi Zou, Yingshuang Ding, Yikang Chen, Xiwu Zhu, Hu Tan, Feiyang Zhang, Chi Wang, Tiancai Zhou, Shuchang Zhang, Li Qi, Xiaojuan Zhao, Hao Yang, Mu Zeng, Wenjun Jin, Xin Computer Vision and Pattern Recognition Generating high-fidelity, controllable, and annotated training data is critical for autonomous driving. Existing methods typically generate a single data form directly from a coarse scene layout, which not only fails to output rich data forms required for diverse downstream tasks but also struggles to model the direct layout-to-data distribution. In this paper, we introduce UniScene, the first unified framework for generating three key data forms - semantic occupancy, video, and LiDAR - in driving scenes. UniScene employs a progressive generation process that decomposes the complex task of scene generation into two hierarchical steps: (a) first generating semantic occupancy from a customized scene layout as a meta scene representation rich in both semantic and geometric information, and then (b) conditioned on occupancy, generating video and LiDAR data, respectively, with two novel transfer strategies of Gaussian-based Joint Rendering and Prior-guided Sparse Modeling. This occupancy-centric approach reduces the generation burden, especially for intricate scenes, while providing detailed intermediate representations for the subsequent generation stages. Extensive experiments demonstrate that UniScene outperforms previous SOTAs in the occupancy, video, and LiDAR generation, which also indeed benefits downstream driving tasks. Project page: https://arlo0o.github.io/uniscene/ |
| title | UniScene: Unified Occupancy-centric Driving Scene Generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2412.05435 |