DaDu-E: Rethinking the Role of Large Language Model in Robotic Computing Pipeline
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916503090102272 |
|---|---|
| author | Sun, Wenhao Hou, Sai Wang, Zixuan Yu, Bo Liu, Shaoshan Yang, Xu Liang, Shuai Gan, Yiming Han, Yinhe |
| author_facet | Sun, Wenhao Hou, Sai Wang, Zixuan Yu, Bo Liu, Shaoshan Yang, Xu Liang, Shuai Gan, Yiming Han, Yinhe |
| contents | Performing complex tasks in open environments remains challenging for robots, even when using large language models (LLMs) as the core planner. Many LLM-based planners are inefficient due to their large number of parameters and prone to inaccuracies because they operate in open-loop systems. We think the reason is that only applying LLMs as planners is insufficient. In this work, we propose DaDu-E, a robust closed-loop planning framework for embodied AI robots. Specifically, DaDu-E is equipped with a relatively lightweight LLM, a set of encapsulated robot skill instructions, a robust feedback system, and memory augmentation. Together, these components enable DaDu-E to (i) actively perceive and adapt to dynamic environments, (ii) optimize computational costs while maintaining high performance, and (iii) recover from execution failures using its memory and feedback mechanisms. Extensive experiments on real-world and simulated tasks show that DaDu-E achieves task success rates comparable to embodied AI robots with larger models as planners like COME-Robot, while reducing computational requirements by $6.6 \times$. Users are encouraged to explore our system at: \url{https://rlc-lab.github.io/dadu-e/}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_01663 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | DaDu-E: Rethinking the Role of Large Language Model in Robotic Computing Pipeline Sun, Wenhao Hou, Sai Wang, Zixuan Yu, Bo Liu, Shaoshan Yang, Xu Liang, Shuai Gan, Yiming Han, Yinhe Robotics Performing complex tasks in open environments remains challenging for robots, even when using large language models (LLMs) as the core planner. Many LLM-based planners are inefficient due to their large number of parameters and prone to inaccuracies because they operate in open-loop systems. We think the reason is that only applying LLMs as planners is insufficient. In this work, we propose DaDu-E, a robust closed-loop planning framework for embodied AI robots. Specifically, DaDu-E is equipped with a relatively lightweight LLM, a set of encapsulated robot skill instructions, a robust feedback system, and memory augmentation. Together, these components enable DaDu-E to (i) actively perceive and adapt to dynamic environments, (ii) optimize computational costs while maintaining high performance, and (iii) recover from execution failures using its memory and feedback mechanisms. Extensive experiments on real-world and simulated tasks show that DaDu-E achieves task success rates comparable to embodied AI robots with larger models as planners like COME-Robot, while reducing computational requirements by $6.6 \times$. Users are encouraged to explore our system at: \url{https://rlc-lab.github.io/dadu-e/}. |
| title | DaDu-E: Rethinking the Role of Large Language Model in Robotic Computing Pipeline |
| topic | Robotics |
| url | https://arxiv.org/abs/2412.01663 |