Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2409.16900 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916616215724032 |
|---|---|
| author | Incao, Sara Mazzola, Carlo Belgiovine, Giulia Sciutti, Alessandra |
| author_facet | Incao, Sara Mazzola, Carlo Belgiovine, Giulia Sciutti, Alessandra |
| contents | The fusion of Large Language Models (LLMs) and robotic systems has led to a transformative paradigm in the robotic field, offering unparalleled capabilities not only in the communication domain but also in skills like multimodal input handling, high-level reasoning, and plan generation. The grounding of LLMs knowledge into the empirical world has been considered a crucial pathway to exploit the efficiency of LLMs in robotics. Nevertheless, connecting LLMs' representations to the external world with multimodal approaches or with robots' bodies is not enough to let them understand the meaning of the language they are manipulating. Taking inspiration from humans, this work draws attention to three necessary elements for an agent to grasp and experience the world. The roadmap for LLMs grounding is envisaged in an active bodily system as the reference point for experiencing the environment, a temporally structured experience for a coherent, self-related interaction with the external world, and social skills to acquire a common-grounded shared experience. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_16900 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Roadmap for Embodied and Social Grounding in LLMs Incao, Sara Mazzola, Carlo Belgiovine, Giulia Sciutti, Alessandra Robotics Artificial Intelligence Computation and Language Human-Computer Interaction I.2.7; I.2.9; J.4; F.3.2; D.3.1 The fusion of Large Language Models (LLMs) and robotic systems has led to a transformative paradigm in the robotic field, offering unparalleled capabilities not only in the communication domain but also in skills like multimodal input handling, high-level reasoning, and plan generation. The grounding of LLMs knowledge into the empirical world has been considered a crucial pathway to exploit the efficiency of LLMs in robotics. Nevertheless, connecting LLMs' representations to the external world with multimodal approaches or with robots' bodies is not enough to let them understand the meaning of the language they are manipulating. Taking inspiration from humans, this work draws attention to three necessary elements for an agent to grasp and experience the world. The roadmap for LLMs grounding is envisaged in an active bodily system as the reference point for experiencing the environment, a temporally structured experience for a coherent, self-related interaction with the external world, and social skills to acquire a common-grounded shared experience. |
| title | A Roadmap for Embodied and Social Grounding in LLMs |
| topic | Robotics Artificial Intelligence Computation and Language Human-Computer Interaction I.2.7; I.2.9; J.4; F.3.2; D.3.1 |
| url | https://arxiv.org/abs/2409.16900 |