MTDrive: Multi-turn Interactive Reinforcement Learning for Autonomous Driving
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911411143180288 |
|---|---|
| author | Li, Xidong Guo, Mingyu Xu, Chenchao Li, Bailin Zhu, Wenjing Zou, Yangang Chen, Rui Wang, Zehuan |
| author_facet | Li, Xidong Guo, Mingyu Xu, Chenchao Li, Bailin Zhu, Wenjing Zou, Yangang Chen, Rui Wang, Zehuan |
| contents | Trajectory planning is a core task in autonomous driving, requiring the prediction of safe and comfortable paths across diverse scenarios. Integrating Multi-modal Large Language Models (MLLMs) with Reinforcement Learning (RL) has shown promise in addressing "long-tail" scenarios. However, existing methods are constrained to single-turn reasoning, limiting their ability to handle complex tasks requiring iterative refinement. To overcome this limitation, we present MTDrive, a multi-turn framework that enables MLLMs to iteratively refine trajectories based on environmental feedback. MTDrive introduces Multi-Turn Group Relative Policy Optimization (mtGRPO), which mitigates reward sparsity by computing relative advantages across turns. We further construct an interactive trajectory understanding dataset from closed-loop simulation to support multi-turn training. Experiments on the NAVSIM benchmark demonstrate superior performance compared to existing methods, validating the effectiveness of our multi-turn reasoning paradigm. Additionally, we implement system-level optimizations to reduce data transfer overhead caused by high-resolution images and multi-turn sequences, achieving 2.5x training throughput. Our data, models, and code will be made available soon. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_22930 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | MTDrive: Multi-turn Interactive Reinforcement Learning for Autonomous Driving Li, Xidong Guo, Mingyu Xu, Chenchao Li, Bailin Zhu, Wenjing Zou, Yangang Chen, Rui Wang, Zehuan Robotics Artificial Intelligence Machine Learning I.2.9; I.2.6 Trajectory planning is a core task in autonomous driving, requiring the prediction of safe and comfortable paths across diverse scenarios. Integrating Multi-modal Large Language Models (MLLMs) with Reinforcement Learning (RL) has shown promise in addressing "long-tail" scenarios. However, existing methods are constrained to single-turn reasoning, limiting their ability to handle complex tasks requiring iterative refinement. To overcome this limitation, we present MTDrive, a multi-turn framework that enables MLLMs to iteratively refine trajectories based on environmental feedback. MTDrive introduces Multi-Turn Group Relative Policy Optimization (mtGRPO), which mitigates reward sparsity by computing relative advantages across turns. We further construct an interactive trajectory understanding dataset from closed-loop simulation to support multi-turn training. Experiments on the NAVSIM benchmark demonstrate superior performance compared to existing methods, validating the effectiveness of our multi-turn reasoning paradigm. Additionally, we implement system-level optimizations to reduce data transfer overhead caused by high-resolution images and multi-turn sequences, achieving 2.5x training throughput. Our data, models, and code will be made available soon. |
| title | MTDrive: Multi-turn Interactive Reinforcement Learning for Autonomous Driving |
| topic | Robotics Artificial Intelligence Machine Learning I.2.9; I.2.6 |
| url | https://arxiv.org/abs/2601.22930 |