Text2World: Benchmarking Large Language Models for Symbolic World Model Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915169285701632 |
|---|---|
| author | Hu, Mengkang Chen, Tianxing Zou, Yude Lei, Yuheng Chen, Qiguang Li, Ming Mu, Yao Zhang, Hongyuan Shao, Wenqi Luo, Ping |
| author_facet | Hu, Mengkang Chen, Tianxing Zou, Yude Lei, Yuheng Chen, Qiguang Li, Ming Mu, Yao Zhang, Hongyuan Shao, Wenqi Luo, Ping |
| contents | Recently, there has been growing interest in leveraging large language models (LLMs) to generate symbolic world models from textual descriptions. Although LLMs have been extensively explored in the context of world modeling, prior studies encountered several challenges, including evaluation randomness, dependence on indirect metrics, and a limited domain scope. To address these limitations, we introduce a novel benchmark, Text2World, based on planning domain definition language (PDDL), featuring hundreds of diverse domains and employing multi-criteria, execution-based metrics for a more robust evaluation. We benchmark current LLMs using Text2World and find that reasoning models trained with large-scale reinforcement learning outperform others. However, even the best-performing model still demonstrates limited capabilities in world modeling. Building on these insights, we examine several promising strategies to enhance the world modeling capabilities of LLMs, including test-time scaling, agent training, and more. We hope that Text2World can serve as a crucial resource, laying the groundwork for future research in leveraging LLMs as world models. The project page is available at https://text-to-world.github.io/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_13092 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Text2World: Benchmarking Large Language Models for Symbolic World Model Generation Hu, Mengkang Chen, Tianxing Zou, Yude Lei, Yuheng Chen, Qiguang Li, Ming Mu, Yao Zhang, Hongyuan Shao, Wenqi Luo, Ping Computation and Language Artificial Intelligence Recently, there has been growing interest in leveraging large language models (LLMs) to generate symbolic world models from textual descriptions. Although LLMs have been extensively explored in the context of world modeling, prior studies encountered several challenges, including evaluation randomness, dependence on indirect metrics, and a limited domain scope. To address these limitations, we introduce a novel benchmark, Text2World, based on planning domain definition language (PDDL), featuring hundreds of diverse domains and employing multi-criteria, execution-based metrics for a more robust evaluation. We benchmark current LLMs using Text2World and find that reasoning models trained with large-scale reinforcement learning outperform others. However, even the best-performing model still demonstrates limited capabilities in world modeling. Building on these insights, we examine several promising strategies to enhance the world modeling capabilities of LLMs, including test-time scaling, agent training, and more. We hope that Text2World can serve as a crucial resource, laying the groundwork for future research in leveraging LLMs as world models. The project page is available at https://text-to-world.github.io/. |
| title | Text2World: Benchmarking Large Language Models for Symbolic World Model Generation |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2502.13092 |