SceneDiffuser: Efficient and Controllable Driving Simulation Initialization and Rollout
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929633874673664 |
|---|---|
| author | Jiang, Chiyu Max Bai, Yijing Cornman, Andre Davis, Christopher Huang, Xiukun Jeon, Hong Kulshrestha, Sakshum Lambert, John Li, Shuangyu Zhou, Xuanyu Fuertes, Carlos Yuan, Chang Tan, Mingxing Zhou, Yin Anguelov, Dragomir |
| author_facet | Jiang, Chiyu Max Bai, Yijing Cornman, Andre Davis, Christopher Huang, Xiukun Jeon, Hong Kulshrestha, Sakshum Lambert, John Li, Shuangyu Zhou, Xuanyu Fuertes, Carlos Yuan, Chang Tan, Mingxing Zhou, Yin Anguelov, Dragomir |
| contents | Realistic and interactive scene simulation is a key prerequisite for autonomous vehicle (AV) development. In this work, we present SceneDiffuser, a scene-level diffusion prior designed for traffic simulation. It offers a unified framework that addresses two key stages of simulation: scene initialization, which involves generating initial traffic layouts, and scene rollout, which encompasses the closed-loop simulation of agent behaviors. While diffusion models have been proven effective in learning realistic and multimodal agent distributions, several challenges remain, including controllability, maintaining realism in closed-loop simulations, and ensuring inference efficiency. To address these issues, we introduce amortized diffusion for simulation. This novel diffusion denoising paradigm amortizes the computational cost of denoising over future simulation steps, significantly reducing the cost per rollout step (16x less inference steps) while also mitigating closed-loop errors. We further enhance controllability through the introduction of generalized hard constraints, a simple yet effective inference-time constraint mechanism, as well as language-based constrained scene generation via few-shot prompting of a large language model (LLM). Our investigations into model scaling reveal that increased computational resources significantly improve overall simulation realism. We demonstrate the effectiveness of our approach on the Waymo Open Sim Agents Challenge, achieving top open-loop performance and the best closed-loop performance among diffusion models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_12129 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | SceneDiffuser: Efficient and Controllable Driving Simulation Initialization and Rollout Jiang, Chiyu Max Bai, Yijing Cornman, Andre Davis, Christopher Huang, Xiukun Jeon, Hong Kulshrestha, Sakshum Lambert, John Li, Shuangyu Zhou, Xuanyu Fuertes, Carlos Yuan, Chang Tan, Mingxing Zhou, Yin Anguelov, Dragomir Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition 68T07 I.2.6 Realistic and interactive scene simulation is a key prerequisite for autonomous vehicle (AV) development. In this work, we present SceneDiffuser, a scene-level diffusion prior designed for traffic simulation. It offers a unified framework that addresses two key stages of simulation: scene initialization, which involves generating initial traffic layouts, and scene rollout, which encompasses the closed-loop simulation of agent behaviors. While diffusion models have been proven effective in learning realistic and multimodal agent distributions, several challenges remain, including controllability, maintaining realism in closed-loop simulations, and ensuring inference efficiency. To address these issues, we introduce amortized diffusion for simulation. This novel diffusion denoising paradigm amortizes the computational cost of denoising over future simulation steps, significantly reducing the cost per rollout step (16x less inference steps) while also mitigating closed-loop errors. We further enhance controllability through the introduction of generalized hard constraints, a simple yet effective inference-time constraint mechanism, as well as language-based constrained scene generation via few-shot prompting of a large language model (LLM). Our investigations into model scaling reveal that increased computational resources significantly improve overall simulation realism. We demonstrate the effectiveness of our approach on the Waymo Open Sim Agents Challenge, achieving top open-loop performance and the best closed-loop performance among diffusion models. |
| title | SceneDiffuser: Efficient and Controllable Driving Simulation Initialization and Rollout |
| topic | Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition 68T07 I.2.6 |
| url | https://arxiv.org/abs/2412.12129 |