Drive&Gen: Co-Evaluating End-to-End Driving and Video Generation Models
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909830144327680 |
|---|---|
| author | Wang, Jiahao Yang, Zhenpei Bai, Yijing Li, Yingwei Zou, Yuliang Sun, Bo Kundu, Abhijit Lezama, Jose Huang, Luna Yue Zhu, Zehao Hwang, Jyh-Jing Anguelov, Dragomir Tan, Mingxing Jiang, Chiyu Max |
| author_facet | Wang, Jiahao Yang, Zhenpei Bai, Yijing Li, Yingwei Zou, Yuliang Sun, Bo Kundu, Abhijit Lezama, Jose Huang, Luna Yue Zhu, Zehao Hwang, Jyh-Jing Anguelov, Dragomir Tan, Mingxing Jiang, Chiyu Max |
| contents | Recent advances in generative models have sparked exciting new possibilities in the field of autonomous vehicles. Specifically, video generation models are now being explored as controllable virtual testing environments. Simultaneously, end-to-end (E2E) driving models have emerged as a streamlined alternative to conventional modular autonomous driving systems, gaining popularity for their simplicity and scalability. However, the application of these techniques to simulation and planning raises important questions. First, while video generation models can generate increasingly realistic videos, can these videos faithfully adhere to the specified conditions and be realistic enough for E2E autonomous planner evaluation? Second, given that data is crucial for understanding and controlling E2E planners, how can we gain deeper insights into their biases and improve their ability to generalize to out-of-distribution scenarios? In this work, we bridge the gap between the driving models and generative world models (Drive&Gen) to address these questions. We propose novel statistical measures leveraging E2E drivers to evaluate the realism of generated videos. By exploiting the controllability of the video generation model, we conduct targeted experiments to investigate distribution gaps affecting E2E planner performance. Finally, we show that synthetic data produced by the video generation model offers a cost-effective alternative to real-world data collection. This synthetic data effectively improves E2E model generalization beyond existing Operational Design Domains, facilitating the expansion of autonomous vehicle services into new operational contexts. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_06209 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Drive&Gen: Co-Evaluating End-to-End Driving and Video Generation Models Wang, Jiahao Yang, Zhenpei Bai, Yijing Li, Yingwei Zou, Yuliang Sun, Bo Kundu, Abhijit Lezama, Jose Huang, Luna Yue Zhu, Zehao Hwang, Jyh-Jing Anguelov, Dragomir Tan, Mingxing Jiang, Chiyu Max Computer Vision and Pattern Recognition Recent advances in generative models have sparked exciting new possibilities in the field of autonomous vehicles. Specifically, video generation models are now being explored as controllable virtual testing environments. Simultaneously, end-to-end (E2E) driving models have emerged as a streamlined alternative to conventional modular autonomous driving systems, gaining popularity for their simplicity and scalability. However, the application of these techniques to simulation and planning raises important questions. First, while video generation models can generate increasingly realistic videos, can these videos faithfully adhere to the specified conditions and be realistic enough for E2E autonomous planner evaluation? Second, given that data is crucial for understanding and controlling E2E planners, how can we gain deeper insights into their biases and improve their ability to generalize to out-of-distribution scenarios? In this work, we bridge the gap between the driving models and generative world models (Drive&Gen) to address these questions. We propose novel statistical measures leveraging E2E drivers to evaluate the realism of generated videos. By exploiting the controllability of the video generation model, we conduct targeted experiments to investigate distribution gaps affecting E2E planner performance. Finally, we show that synthetic data produced by the video generation model offers a cost-effective alternative to real-world data collection. This synthetic data effectively improves E2E model generalization beyond existing Operational Design Domains, facilitating the expansion of autonomous vehicle services into new operational contexts. |
| title | Drive&Gen: Co-Evaluating End-to-End Driving and Video Generation Models |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2510.06209 |