Precise Drive with VLM: First Prize Solution for PRCV 2024 Drive LM challenge
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909377916567552 |
|---|---|
| author | Huang, Bin Wang, Siyu Chen, Yuanpeng Wu, Yidan Song, Hui Ding, Zifan Leng, Jing Liang, Chengpeng Xue, Peng Zhang, Junliang Zhao, Tiankun |
| author_facet | Huang, Bin Wang, Siyu Chen, Yuanpeng Wu, Yidan Song, Hui Ding, Zifan Leng, Jing Liang, Chengpeng Xue, Peng Zhang, Junliang Zhao, Tiankun |
| contents | This technical report outlines the methodologies we applied for the PRCV Challenge, focusing on cognition and decision-making in driving scenarios. We employed InternVL-2.0, a pioneering open-source multi-modal model, and enhanced it by refining both the model input and training methodologies. For the input data, we strategically concatenated and formatted the multi-view images. It is worth mentioning that we utilized the coordinates of the original images without transformation. In terms of model training, we initially pre-trained the model on publicly available autonomous driving scenario datasets to bolster its alignment capabilities of the challenge tasks, followed by fine-tuning on the DriveLM-nuscenes Dataset. During the fine-tuning phase, we innovatively modified the loss function to enhance the model's precision in predicting coordinate values. These approaches ensure that our model possesses advanced cognitive and decision-making capabilities in driving scenarios. Consequently, our model achieved a score of 0.6064, securing the first prize on the competition's final results. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_02999 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Precise Drive with VLM: First Prize Solution for PRCV 2024 Drive LM challenge Huang, Bin Wang, Siyu Chen, Yuanpeng Wu, Yidan Song, Hui Ding, Zifan Leng, Jing Liang, Chengpeng Xue, Peng Zhang, Junliang Zhao, Tiankun Computer Vision and Pattern Recognition This technical report outlines the methodologies we applied for the PRCV Challenge, focusing on cognition and decision-making in driving scenarios. We employed InternVL-2.0, a pioneering open-source multi-modal model, and enhanced it by refining both the model input and training methodologies. For the input data, we strategically concatenated and formatted the multi-view images. It is worth mentioning that we utilized the coordinates of the original images without transformation. In terms of model training, we initially pre-trained the model on publicly available autonomous driving scenario datasets to bolster its alignment capabilities of the challenge tasks, followed by fine-tuning on the DriveLM-nuscenes Dataset. During the fine-tuning phase, we innovatively modified the loss function to enhance the model's precision in predicting coordinate values. These approaches ensure that our model possesses advanced cognitive and decision-making capabilities in driving scenarios. Consequently, our model achieved a score of 0.6064, securing the first prize on the competition's final results. |
| title | Precise Drive with VLM: First Prize Solution for PRCV 2024 Drive LM challenge |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2411.02999 |