Impromptu VLA: Open Weights and Open Data for Driving Vision-Language-Action Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909627501772800 |
|---|---|
| author | Chi, Haohan Gao, Huan-ang Liu, Ziming Liu, Jianing Liu, Chenyu Li, Jinwei Yang, Kaisen Yu, Yangcheng Wang, Zeda Li, Wenyi Wang, Leichen Hu, Xingtao Sun, Hao Zhao, Hang Zhao, Hao |
| author_facet | Chi, Haohan Gao, Huan-ang Liu, Ziming Liu, Jianing Liu, Chenyu Li, Jinwei Yang, Kaisen Yu, Yangcheng Wang, Zeda Li, Wenyi Wang, Leichen Hu, Xingtao Sun, Hao Zhao, Hang Zhao, Hao |
| contents | Vision-Language-Action (VLA) models for autonomous driving show promise but falter in unstructured corner case scenarios, largely due to a scarcity of targeted benchmarks. To address this, we introduce Impromptu VLA. Our core contribution is the Impromptu VLA Dataset: over 80,000 meticulously curated video clips, distilled from over 2M source clips sourced from 8 open-source large-scale datasets. This dataset is built upon our novel taxonomy of four challenging unstructured categories and features rich, planning-oriented question-answering annotations and action trajectories. Crucially, experiments demonstrate that VLAs trained with our dataset achieve substantial performance gains on established benchmarks--improving closed-loop NeuroNCAP scores and collision rates, and reaching near state-of-the-art L2 accuracy in open-loop nuScenes trajectory prediction. Furthermore, our Q&A suite serves as an effective diagnostic, revealing clear VLM improvements in perception, prediction, and planning. Our code, data and models are available at https://github.com/ahydchh/Impromptu-VLA. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_23757 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Impromptu VLA: Open Weights and Open Data for Driving Vision-Language-Action Models Chi, Haohan Gao, Huan-ang Liu, Ziming Liu, Jianing Liu, Chenyu Li, Jinwei Yang, Kaisen Yu, Yangcheng Wang, Zeda Li, Wenyi Wang, Leichen Hu, Xingtao Sun, Hao Zhao, Hang Zhao, Hao Computer Vision and Pattern Recognition Vision-Language-Action (VLA) models for autonomous driving show promise but falter in unstructured corner case scenarios, largely due to a scarcity of targeted benchmarks. To address this, we introduce Impromptu VLA. Our core contribution is the Impromptu VLA Dataset: over 80,000 meticulously curated video clips, distilled from over 2M source clips sourced from 8 open-source large-scale datasets. This dataset is built upon our novel taxonomy of four challenging unstructured categories and features rich, planning-oriented question-answering annotations and action trajectories. Crucially, experiments demonstrate that VLAs trained with our dataset achieve substantial performance gains on established benchmarks--improving closed-loop NeuroNCAP scores and collision rates, and reaching near state-of-the-art L2 accuracy in open-loop nuScenes trajectory prediction. Furthermore, our Q&A suite serves as an effective diagnostic, revealing clear VLM improvements in perception, prediction, and planning. Our code, data and models are available at https://github.com/ahydchh/Impromptu-VLA. |
| title | Impromptu VLA: Open Weights and Open Data for Driving Vision-Language-Action Models |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2505.23757 |