Natural Language as Policies: Reasoning for Coordinate-Level Embodied Control with LLMs
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916195870965760 |
|---|---|
| author | Mikami, Yusuke Melnik, Andrew Miura, Jun Hautamäki, Ville |
| author_facet | Mikami, Yusuke Melnik, Andrew Miura, Jun Hautamäki, Ville |
| contents | We demonstrate experimental results with LLMs that address robotics task planning problems. Recently, LLMs have been applied in robotics task planning, particularly using a code generation approach that converts complex high-level instructions into mid-level policy codes. In contrast, our approach acquires text descriptions of the task and scene objects, then formulates task planning through natural language reasoning, and outputs coordinate level control commands, thus reducing the necessity for intermediate representation code as policies with pre-defined APIs. Our approach is evaluated on a multi-modal prompt simulation benchmark, demonstrating that our prompt engineering experiments with natural language reasoning significantly enhance success rates compared to its absence. Furthermore, our approach illustrates the potential for natural language descriptions to transfer robotics skills from known tasks to previously unseen tasks. The project website: https://natural-language-as-policies.github.io/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_13801 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Natural Language as Policies: Reasoning for Coordinate-Level Embodied Control with LLMs Mikami, Yusuke Melnik, Andrew Miura, Jun Hautamäki, Ville Robotics Artificial Intelligence Computation and Language I.2.9; I.2.7 We demonstrate experimental results with LLMs that address robotics task planning problems. Recently, LLMs have been applied in robotics task planning, particularly using a code generation approach that converts complex high-level instructions into mid-level policy codes. In contrast, our approach acquires text descriptions of the task and scene objects, then formulates task planning through natural language reasoning, and outputs coordinate level control commands, thus reducing the necessity for intermediate representation code as policies with pre-defined APIs. Our approach is evaluated on a multi-modal prompt simulation benchmark, demonstrating that our prompt engineering experiments with natural language reasoning significantly enhance success rates compared to its absence. Furthermore, our approach illustrates the potential for natural language descriptions to transfer robotics skills from known tasks to previously unseen tasks. The project website: https://natural-language-as-policies.github.io/ |
| title | Natural Language as Policies: Reasoning for Coordinate-Level Embodied Control with LLMs |
| topic | Robotics Artificial Intelligence Computation and Language I.2.9; I.2.7 |
| url | https://arxiv.org/abs/2403.13801 |