PLANET: A Collection of Benchmarks for Evaluating LLMs' Planning Capabilities
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915251661832192 |
|---|---|
| author | Li, Haoming Chen, Zhaoliang Zhang, Jonathan Liu, Fei |
| author_facet | Li, Haoming Chen, Zhaoliang Zhang, Jonathan Liu, Fei |
| contents | Planning is central to agents and agentic AI. The ability to plan, e.g., creating travel itineraries within a budget, holds immense potential in both scientific and commercial contexts. Moreover, optimal plans tend to require fewer resources compared to ad-hoc methods. To date, a comprehensive understanding of existing planning benchmarks appears to be lacking. Without it, comparing planning algorithms' performance across domains or selecting suitable algorithms for new scenarios remains challenging. In this paper, we examine a range of planning benchmarks to identify commonly used testbeds for algorithm development and highlight potential gaps. These benchmarks are categorized into embodied environments, web navigation, scheduling, games and puzzles, and everyday task automation. Our study recommends the most appropriate benchmarks for various algorithms and offers insights to guide future benchmark development. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_14773 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | PLANET: A Collection of Benchmarks for Evaluating LLMs' Planning Capabilities Li, Haoming Chen, Zhaoliang Zhang, Jonathan Liu, Fei Artificial Intelligence Computation and Language Machine Learning Multiagent Systems Planning is central to agents and agentic AI. The ability to plan, e.g., creating travel itineraries within a budget, holds immense potential in both scientific and commercial contexts. Moreover, optimal plans tend to require fewer resources compared to ad-hoc methods. To date, a comprehensive understanding of existing planning benchmarks appears to be lacking. Without it, comparing planning algorithms' performance across domains or selecting suitable algorithms for new scenarios remains challenging. In this paper, we examine a range of planning benchmarks to identify commonly used testbeds for algorithm development and highlight potential gaps. These benchmarks are categorized into embodied environments, web navigation, scheduling, games and puzzles, and everyday task automation. Our study recommends the most appropriate benchmarks for various algorithms and offers insights to guide future benchmark development. |
| title | PLANET: A Collection of Benchmarks for Evaluating LLMs' Planning Capabilities |
| topic | Artificial Intelligence Computation and Language Machine Learning Multiagent Systems |
| url | https://arxiv.org/abs/2504.14773 |