DrivingDojo Dataset: Advancing Interactive and Knowledge-Enriched Driving World Model
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929541378736128 |
|---|---|
| author | Wang, Yuqi Cheng, Ke He, Jiawei Wang, Qitai Dai, Hengchen Chen, Yuntao Xia, Fei Zhang, Zhaoxiang |
| author_facet | Wang, Yuqi Cheng, Ke He, Jiawei Wang, Qitai Dai, Hengchen Chen, Yuntao Xia, Fei Zhang, Zhaoxiang |
| contents | Driving world models have gained increasing attention due to their ability to model complex physical dynamics. However, their superb modeling capability is yet to be fully unleashed due to the limited video diversity in current driving datasets. We introduce DrivingDojo, the first dataset tailor-made for training interactive world models with complex driving dynamics. Our dataset features video clips with a complete set of driving maneuvers, diverse multi-agent interplay, and rich open-world driving knowledge, laying a stepping stone for future world model development. We further define an action instruction following (AIF) benchmark for world models and demonstrate the superiority of the proposed dataset for generating action-controlled future predictions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_10738 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | DrivingDojo Dataset: Advancing Interactive and Knowledge-Enriched Driving World Model Wang, Yuqi Cheng, Ke He, Jiawei Wang, Qitai Dai, Hengchen Chen, Yuntao Xia, Fei Zhang, Zhaoxiang Computer Vision and Pattern Recognition Artificial Intelligence Driving world models have gained increasing attention due to their ability to model complex physical dynamics. However, their superb modeling capability is yet to be fully unleashed due to the limited video diversity in current driving datasets. We introduce DrivingDojo, the first dataset tailor-made for training interactive world models with complex driving dynamics. Our dataset features video clips with a complete set of driving maneuvers, diverse multi-agent interplay, and rich open-world driving knowledge, laying a stepping stone for future world model development. We further define an action instruction following (AIF) benchmark for world models and demonstrate the superiority of the proposed dataset for generating action-controlled future predictions. |
| title | DrivingDojo Dataset: Advancing Interactive and Knowledge-Enriched Driving World Model |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2410.10738 |