Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2601.20856 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918311889993728 |
|---|---|
| author | Monti, Sebastiano Nicolini, Carlo Pellegrini, Gianni Staiano, Jacopo Lepri, Bruno |
| author_facet | Monti, Sebastiano Nicolini, Carlo Pellegrini, Gianni Staiano, Jacopo Lepri, Bruno |
| contents | Although the capabilities of large language models have been increasingly tested on complex reasoning tasks, their long-horizon planning abilities have not yet been extensively investigated. In this work, we provide a systematic assessment of the planning and long-horizon reasoning capabilities of state-of-the-art Large Reasoning Models (LRMs). We propose a novel benchmark based on Sokoban puzzles, intentionally simplified to isolate long-horizon planning from state persistence. Our findings reveal a consistent degradation in planning performance when more than 25 moves are required to reach the solution, suggesting a fundamental constraint on forward planning capacity. We show that equipping LRMs with Planning Domain Definition Language (PDDL) parsing, validation, and solving tools allows for modest improvements, suggesting inherent architectural limitations which might not be overcome by test-time scaling approaches alone. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_20856 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SokoBench: Evaluating Long-Horizon Planning and Reasoning in Large Language Models Monti, Sebastiano Nicolini, Carlo Pellegrini, Gianni Staiano, Jacopo Lepri, Bruno Artificial Intelligence Although the capabilities of large language models have been increasingly tested on complex reasoning tasks, their long-horizon planning abilities have not yet been extensively investigated. In this work, we provide a systematic assessment of the planning and long-horizon reasoning capabilities of state-of-the-art Large Reasoning Models (LRMs). We propose a novel benchmark based on Sokoban puzzles, intentionally simplified to isolate long-horizon planning from state persistence. Our findings reveal a consistent degradation in planning performance when more than 25 moves are required to reach the solution, suggesting a fundamental constraint on forward planning capacity. We show that equipping LRMs with Planning Domain Definition Language (PDDL) parsing, validation, and solving tools allows for modest improvements, suggesting inherent architectural limitations which might not be overcome by test-time scaling approaches alone. |
| title | SokoBench: Evaluating Long-Horizon Planning and Reasoning in Large Language Models |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2601.20856 |