Understanding Generative AI in Robot Logic Parametrization
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910687161221120 |
|---|---|
| author | Hwang, Yuna Sato, Arissa J. Praveena, Pragathi White, Nathan Thomas Mutlu, Bilge |
| author_facet | Hwang, Yuna Sato, Arissa J. Praveena, Pragathi White, Nathan Thomas Mutlu, Bilge |
| contents | Leveraging generative AI (for example, Large Language Models) for language understanding within robotics opens up possibilities for LLM-driven robot end-user development (EUD). Despite the numerous design opportunities it provides, little is understood about how this technology can be utilized when constructing robot program logic. In this paper, we outline the background in capturing natural language end-user intent and summarize previous use cases of LLMs within EUD. Taking the context of filmmaking as an example, we explore how a cinematography practitioner's intent to film a certain scene can be articulated using natural language, captured by an LLM, and further parametrized as low-level robot arm movement. We explore the capabilities of an LLM interpreting end-user intent and mapping natural language to predefined, cross-modal data in the process of iterative program development. We conclude by suggesting future opportunities for domain exploration beyond cinematography to support language-driven robotic camera navigation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_04273 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Understanding Generative AI in Robot Logic Parametrization Hwang, Yuna Sato, Arissa J. Praveena, Pragathi White, Nathan Thomas Mutlu, Bilge Robotics Human-Computer Interaction I.2.9; J.7 Leveraging generative AI (for example, Large Language Models) for language understanding within robotics opens up possibilities for LLM-driven robot end-user development (EUD). Despite the numerous design opportunities it provides, little is understood about how this technology can be utilized when constructing robot program logic. In this paper, we outline the background in capturing natural language end-user intent and summarize previous use cases of LLMs within EUD. Taking the context of filmmaking as an example, we explore how a cinematography practitioner's intent to film a certain scene can be articulated using natural language, captured by an LLM, and further parametrized as low-level robot arm movement. We explore the capabilities of an LLM interpreting end-user intent and mapping natural language to predefined, cross-modal data in the process of iterative program development. We conclude by suggesting future opportunities for domain exploration beyond cinematography to support language-driven robotic camera navigation. |
| title | Understanding Generative AI in Robot Logic Parametrization |
| topic | Robotics Human-Computer Interaction I.2.9; J.7 |
| url | https://arxiv.org/abs/2411.04273 |