GridRoute: A Benchmark for LLM-Based Route Planning with Cardinal Movement in Grid Environments
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911103235129344 |
|---|---|
| author | Li, Kechen Tao, Yaotian Wen, Ximing Sun, Quanwei Gong, Zifei Xu, Chang Zhang, Xizhe Ji, Tianbo |
| author_facet | Li, Kechen Tao, Yaotian Wen, Ximing Sun, Quanwei Gong, Zifei Xu, Chang Zhang, Xizhe Ji, Tianbo |
| contents | Recent advancements in Large Language Models (LLMs) have demonstrated their potential in planning and reasoning tasks, offering a flexible alternative to classical pathfinding algorithms. However, most existing studies focus on LLMs' independent reasoning capabilities and overlook the potential synergy between LLMs and traditional algorithms. To fill this gap, we propose a comprehensive evaluation benchmark GridRoute to assess how LLMs can take advantage of traditional algorithms. We also propose a novel hybrid prompting technique called Algorithm of Thought (AoT), which introduces traditional algorithms' guidance into prompting. Our benchmark evaluates six LLMs ranging from 7B to 72B parameters across various map sizes, assessing their performance in correctness, optimality, and efficiency in grid environments with varying sizes. Our results show that AoT significantly boosts performance across all model sizes, particularly in larger or more complex environments, suggesting a promising approach to addressing path planning challenges. Our code is open-sourced at https://github.com/LinChance/GridRoute. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_24306 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | GridRoute: A Benchmark for LLM-Based Route Planning with Cardinal Movement in Grid Environments Li, Kechen Tao, Yaotian Wen, Ximing Sun, Quanwei Gong, Zifei Xu, Chang Zhang, Xizhe Ji, Tianbo Artificial Intelligence Recent advancements in Large Language Models (LLMs) have demonstrated their potential in planning and reasoning tasks, offering a flexible alternative to classical pathfinding algorithms. However, most existing studies focus on LLMs' independent reasoning capabilities and overlook the potential synergy between LLMs and traditional algorithms. To fill this gap, we propose a comprehensive evaluation benchmark GridRoute to assess how LLMs can take advantage of traditional algorithms. We also propose a novel hybrid prompting technique called Algorithm of Thought (AoT), which introduces traditional algorithms' guidance into prompting. Our benchmark evaluates six LLMs ranging from 7B to 72B parameters across various map sizes, assessing their performance in correctness, optimality, and efficiency in grid environments with varying sizes. Our results show that AoT significantly boosts performance across all model sizes, particularly in larger or more complex environments, suggesting a promising approach to addressing path planning challenges. Our code is open-sourced at https://github.com/LinChance/GridRoute. |
| title | GridRoute: A Benchmark for LLM-Based Route Planning with Cardinal Movement in Grid Environments |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2505.24306 |