Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2605.24004 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918519221780480 |
|---|---|
| author | Sun, Zhengqi Sun, Yiwen Liu, Boxuan Chen, Tailai Guo, Tianxu Liu, Jiabin |
| author_facet | Sun, Zhengqi Sun, Yiwen Liu, Boxuan Chen, Tailai Guo, Tianxu Liu, Jiabin |
| contents | Large language models (LLMs) are promising for autonomous driving, but semantics-only decision policies can yield physically unsafe behavior in dynamic traffic. Existing methods either perform online language reasoning without explicit dynamics verification or use world models mainly in offline pipelines, leaving a gap between semantic intent and physical feasibility at decision time. We propose Reason--Imagine--Act (RIA), a closed-loop framework that couples an LLM reasoner with an action-conditioned world model for online safety verification. At each step, the LLM proposes an action template and candidate sub-actions, the world model performs short-horizon rollouts, and a safety scorer selects the safest executable action with feedback to the next reasoning step. Under a unified CARLA point-goal protocol (1000 episodes), RIA achieves 80.05% route completion, 51.10% arrival rate, and 0.20% collision rate. Under the same closed-loop interface, RIA consistently outperforms training-free baselines, including CARLA TM and MADA, on core closed-loop metrics. For reproducibility, code is available at https://github.com/pku-smart-city/source_code/tree/main/RIA. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_24004 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Reason--Imagine--Act: Closed-Loop LLM Decision Making with World Models for Autonomous Driving Sun, Zhengqi Sun, Yiwen Liu, Boxuan Chen, Tailai Guo, Tianxu Liu, Jiabin Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning Robotics Large language models (LLMs) are promising for autonomous driving, but semantics-only decision policies can yield physically unsafe behavior in dynamic traffic. Existing methods either perform online language reasoning without explicit dynamics verification or use world models mainly in offline pipelines, leaving a gap between semantic intent and physical feasibility at decision time. We propose Reason--Imagine--Act (RIA), a closed-loop framework that couples an LLM reasoner with an action-conditioned world model for online safety verification. At each step, the LLM proposes an action template and candidate sub-actions, the world model performs short-horizon rollouts, and a safety scorer selects the safest executable action with feedback to the next reasoning step. Under a unified CARLA point-goal protocol (1000 episodes), RIA achieves 80.05% route completion, 51.10% arrival rate, and 0.20% collision rate. Under the same closed-loop interface, RIA consistently outperforms training-free baselines, including CARLA TM and MADA, on core closed-loop metrics. For reproducibility, code is available at https://github.com/pku-smart-city/source_code/tree/main/RIA. |
| title | Reason--Imagine--Act: Closed-Loop LLM Decision Making with World Models for Autonomous Driving |
| topic | Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning Robotics |
| url | https://arxiv.org/abs/2605.24004 |