Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2601.03519 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917195621072896 |
|---|---|
| author | Zhang, Liangdong Nie, Yiming Li, Haoyang Kong, Fanjie Zhang, Baobao Huang, Shunxin Fu, Kai Min, Chen Xiao, Liang |
| author_facet | Zhang, Liangdong Nie, Yiming Li, Haoyang Kong, Fanjie Zhang, Baobao Huang, Shunxin Fu, Kai Min, Chen Xiao, Liang |
| contents | Efficient trajectory planning in off-road terrains presents a formidable challenge for autonomous vehicles, often necessitating complex multi-step pipelines. However, traditional approaches exhibit limited adaptability in dynamic environments. To address these limitations, this paper proposes OFF-EMMA, a novel end-to-end multimodal framework designed to overcome the deficiencies of insufficient spatial perception and unstable reasoning in visual-language-action (VLA) models for off-road autonomous driving scenarios. The framework explicitly annotates input images through the design of a visual prompt block and introduces a chain-of-thought with self-consistency (COT-SC) reasoning strategy to enhance the accuracy and robustness of trajectory planning. The visual prompt block utilizes semantic segmentation masks as visual prompts, enhancing the spatial understanding ability of pre-trained visual-language models for complex terrains. The COT- SC strategy effectively mitigates the error impact of outliers on planning performance through a multi-path reasoning mechanism. Experimental results on the RELLIS-3D off-road dataset demonstrate that OFF-EMMA significantly outperforms existing methods, reducing the average L2 error of the Qwen backbone model by 13.3% and decreasing the failure rate from 16.52% to 6.56%. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_03519 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | A Vision-Language-Action Model with Visual Prompt for OFF-Road Autonomous Driving Zhang, Liangdong Nie, Yiming Li, Haoyang Kong, Fanjie Zhang, Baobao Huang, Shunxin Fu, Kai Min, Chen Xiao, Liang Robotics Efficient trajectory planning in off-road terrains presents a formidable challenge for autonomous vehicles, often necessitating complex multi-step pipelines. However, traditional approaches exhibit limited adaptability in dynamic environments. To address these limitations, this paper proposes OFF-EMMA, a novel end-to-end multimodal framework designed to overcome the deficiencies of insufficient spatial perception and unstable reasoning in visual-language-action (VLA) models for off-road autonomous driving scenarios. The framework explicitly annotates input images through the design of a visual prompt block and introduces a chain-of-thought with self-consistency (COT-SC) reasoning strategy to enhance the accuracy and robustness of trajectory planning. The visual prompt block utilizes semantic segmentation masks as visual prompts, enhancing the spatial understanding ability of pre-trained visual-language models for complex terrains. The COT- SC strategy effectively mitigates the error impact of outliers on planning performance through a multi-path reasoning mechanism. Experimental results on the RELLIS-3D off-road dataset demonstrate that OFF-EMMA significantly outperforms existing methods, reducing the average L2 error of the Qwen backbone model by 13.3% and decreasing the failure rate from 16.52% to 6.56%. |
| title | A Vision-Language-Action Model with Visual Prompt for OFF-Road Autonomous Driving |
| topic | Robotics |
| url | https://arxiv.org/abs/2601.03519 |