LimSim++: A Closed-Loop Platform for Deploying Multimodal LLMs in Autonomous Driving
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866929311897878528 |
|---|---|
| author | Fu, Daocheng Lei, Wenjie Wen, Licheng Cai, Pinlong Mao, Song Dou, Min Shi, Botian Qiao, Yu |
| author_facet | Fu, Daocheng Lei, Wenjie Wen, Licheng Cai, Pinlong Mao, Song Dou, Min Shi, Botian Qiao, Yu |
| contents | The emergence of Multimodal Large Language Models ((M)LLMs) has ushered in new avenues in artificial intelligence, particularly for autonomous driving by offering enhanced understanding and reasoning capabilities. This paper introduces LimSim++, an extended version of LimSim designed for the application of (M)LLMs in autonomous driving. Acknowledging the limitations of existing simulation platforms, LimSim++ addresses the need for a long-term closed-loop infrastructure supporting continuous learning and improved generalization in autonomous driving. The platform offers extended-duration, multi-scenario simulations, providing crucial information for (M)LLM-driven vehicles. Users can engage in prompt engineering, model evaluation, and framework enhancement, making LimSim++ a versatile tool for research and practice. This paper additionally introduces a baseline (M)LLM-driven framework, systematically validated through quantitative experiments across diverse scenarios. The open-source resources of LimSim++ are available at: https://pjlab-adg.github.io/limsim-plus/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_01246 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | LimSim++: A Closed-Loop Platform for Deploying Multimodal LLMs in Autonomous Driving Fu, Daocheng Lei, Wenjie Wen, Licheng Cai, Pinlong Mao, Song Dou, Min Shi, Botian Qiao, Yu Robotics Systems and Control The emergence of Multimodal Large Language Models ((M)LLMs) has ushered in new avenues in artificial intelligence, particularly for autonomous driving by offering enhanced understanding and reasoning capabilities. This paper introduces LimSim++, an extended version of LimSim designed for the application of (M)LLMs in autonomous driving. Acknowledging the limitations of existing simulation platforms, LimSim++ addresses the need for a long-term closed-loop infrastructure supporting continuous learning and improved generalization in autonomous driving. The platform offers extended-duration, multi-scenario simulations, providing crucial information for (M)LLM-driven vehicles. Users can engage in prompt engineering, model evaluation, and framework enhancement, making LimSim++ a versatile tool for research and practice. This paper additionally introduces a baseline (M)LLM-driven framework, systematically validated through quantitative experiments across diverse scenarios. The open-source resources of LimSim++ are available at: https://pjlab-adg.github.io/limsim-plus/. |
| title | LimSim++: A Closed-Loop Platform for Deploying Multimodal LLMs in Autonomous Driving |
| topic | Robotics Systems and Control |
| url | https://arxiv.org/abs/2402.01246 |