Latent Chain-of-Thought World Modeling for End-to-End Driving
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866918444761350144 |
|---|---|
| author | Tan, Shuhan Chitta, Kashyap Chen, Yuxiao Tian, Ran You, Yurong Wang, Yan Luo, Wenjie Cao, Yulong Krahenbuhl, Philipp Pavone, Marco Ivanovic, Boris |
| author_facet | Tan, Shuhan Chitta, Kashyap Chen, Yuxiao Tian, Ran You, Yurong Wang, Yan Luo, Wenjie Cao, Yulong Krahenbuhl, Philipp Pavone, Marco Ivanovic, Boris |
| contents | Recent Vision-Language-Action (VLA) models for autonomous driving explore inference-time reasoning as a way to improve driving performance and safety in challenging scenarios. Most prior work uses natural language to express chain-of-thought (CoT) reasoning before producing driving actions. However, text may not be the most efficient representation for reasoning. In this work, we present Latent-CoT-Drive (LCDrive): a model that expresses CoT in a latent language that captures possible outcomes of the driving actions being considered. Our approach unifies CoT reasoning and decision making by representing both in an action-aligned latent space. Instead of natural language, the model reasons by interleaving (1) action-proposal tokens, which use the same vocabulary as the model's output actions; and (2) world model tokens, which are grounded in a learned latent world model and express future outcomes of these actions. We cold start latent CoT by supervising the model's action proposals and world model tokens based on ground-truth future rollouts of the scene. We then post-train with closed-loop reinforcement learning to strengthen reasoning capabilities. On a large-scale end-to-end driving benchmark, LCDrive achieves faster inference, better trajectory quality, and larger improvements from interactive reinforcement learning compared to both non-reasoning and text-reasoning baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_10226 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Latent Chain-of-Thought World Modeling for End-to-End Driving Tan, Shuhan Chitta, Kashyap Chen, Yuxiao Tian, Ran You, Yurong Wang, Yan Luo, Wenjie Cao, Yulong Krahenbuhl, Philipp Pavone, Marco Ivanovic, Boris Computer Vision and Pattern Recognition Robotics Recent Vision-Language-Action (VLA) models for autonomous driving explore inference-time reasoning as a way to improve driving performance and safety in challenging scenarios. Most prior work uses natural language to express chain-of-thought (CoT) reasoning before producing driving actions. However, text may not be the most efficient representation for reasoning. In this work, we present Latent-CoT-Drive (LCDrive): a model that expresses CoT in a latent language that captures possible outcomes of the driving actions being considered. Our approach unifies CoT reasoning and decision making by representing both in an action-aligned latent space. Instead of natural language, the model reasons by interleaving (1) action-proposal tokens, which use the same vocabulary as the model's output actions; and (2) world model tokens, which are grounded in a learned latent world model and express future outcomes of these actions. We cold start latent CoT by supervising the model's action proposals and world model tokens based on ground-truth future rollouts of the scene. We then post-train with closed-loop reinforcement learning to strengthen reasoning capabilities. On a large-scale end-to-end driving benchmark, LCDrive achieves faster inference, better trajectory quality, and larger improvements from interactive reinforcement learning compared to both non-reasoning and text-reasoning baselines. |
| title | Latent Chain-of-Thought World Modeling for End-to-End Driving |
| topic | Computer Vision and Pattern Recognition Robotics |
| url | https://arxiv.org/abs/2512.10226 |