Large Language Models as Planning Domain Generators
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916241975803904 |
|---|---|
| author | Oswald, James Srinivas, Kavitha Kokel, Harsha Lee, Junkyu Katz, Michael Sohrabi, Shirin |
| author_facet | Oswald, James Srinivas, Kavitha Kokel, Harsha Lee, Junkyu Katz, Michael Sohrabi, Shirin |
| contents | Developing domain models is one of the few remaining places that require manual human labor in AI planning. Thus, in order to make planning more accessible, it is desirable to automate the process of domain model generation. To this end, we investigate if large language models (LLMs) can be used to generate planning domain models from simple textual descriptions. Specifically, we introduce a framework for automated evaluation of LLM-generated domains by comparing the sets of plans for domain instances. Finally, we perform an empirical analysis of 7 large language models, including coding and chat models across 9 different planning domains, and under three classes of natural language domain descriptions. Our results indicate that LLMs, particularly those with high parameter counts, exhibit a moderate level of proficiency in generating correct planning domains from natural language descriptions. Our code is available at https://github.com/IBM/NL2PDDL. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_06650 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Large Language Models as Planning Domain Generators Oswald, James Srinivas, Kavitha Kokel, Harsha Lee, Junkyu Katz, Michael Sohrabi, Shirin Computation and Language Artificial Intelligence Developing domain models is one of the few remaining places that require manual human labor in AI planning. Thus, in order to make planning more accessible, it is desirable to automate the process of domain model generation. To this end, we investigate if large language models (LLMs) can be used to generate planning domain models from simple textual descriptions. Specifically, we introduce a framework for automated evaluation of LLM-generated domains by comparing the sets of plans for domain instances. Finally, we perform an empirical analysis of 7 large language models, including coding and chat models across 9 different planning domains, and under three classes of natural language domain descriptions. Our results indicate that LLMs, particularly those with high parameter counts, exhibit a moderate level of proficiency in generating correct planning domains from natural language descriptions. Our code is available at https://github.com/IBM/NL2PDDL. |
| title | Large Language Models as Planning Domain Generators |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2405.06650 |