LaMI: Large Language Models for Multi-Modal Human-Robot Interaction
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911835302658048 |
|---|---|
| author | Wang, Chao Hasler, Stephan Tanneberg, Daniel Ocker, Felix Joublin, Frank Ceravola, Antonello Deigmoeller, Joerg Gienger, Michael |
| author_facet | Wang, Chao Hasler, Stephan Tanneberg, Daniel Ocker, Felix Joublin, Frank Ceravola, Antonello Deigmoeller, Joerg Gienger, Michael |
| contents | This paper presents an innovative large language model (LLM)-based robotic system for enhancing multi-modal human-robot interaction (HRI). Traditional HRI systems relied on complex designs for intent estimation, reasoning, and behavior generation, which were resource-intensive. In contrast, our system empowers researchers and practitioners to regulate robot behavior through three key aspects: providing high-level linguistic guidance, creating "atomic actions" and expressions the robot can use, and offering a set of examples. Implemented on a physical robot, it demonstrates proficiency in adapting to multi-modal inputs and determining the appropriate manner of action to assist humans with its arms, following researchers' defined guidelines. Simultaneously, it coordinates the robot's lid, neck, and ear movements with speech output to produce dynamic, multi-modal expressions. This showcases the system's potential to revolutionize HRI by shifting from conventional, manual state-and-flow design methods to an intuitive, guidance-based, and example-driven approach. Supplementary material can be found at https://hri-eu.github.io/Lami/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_15174 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | LaMI: Large Language Models for Multi-Modal Human-Robot Interaction Wang, Chao Hasler, Stephan Tanneberg, Daniel Ocker, Felix Joublin, Frank Ceravola, Antonello Deigmoeller, Joerg Gienger, Michael Robotics Human-Computer Interaction This paper presents an innovative large language model (LLM)-based robotic system for enhancing multi-modal human-robot interaction (HRI). Traditional HRI systems relied on complex designs for intent estimation, reasoning, and behavior generation, which were resource-intensive. In contrast, our system empowers researchers and practitioners to regulate robot behavior through three key aspects: providing high-level linguistic guidance, creating "atomic actions" and expressions the robot can use, and offering a set of examples. Implemented on a physical robot, it demonstrates proficiency in adapting to multi-modal inputs and determining the appropriate manner of action to assist humans with its arms, following researchers' defined guidelines. Simultaneously, it coordinates the robot's lid, neck, and ear movements with speech output to produce dynamic, multi-modal expressions. This showcases the system's potential to revolutionize HRI by shifting from conventional, manual state-and-flow design methods to an intuitive, guidance-based, and example-driven approach. Supplementary material can be found at https://hri-eu.github.io/Lami/ |
| title | LaMI: Large Language Models for Multi-Modal Human-Robot Interaction |
| topic | Robotics Human-Computer Interaction |
| url | https://arxiv.org/abs/2401.15174 |