DaDu-Corki: Algorithm-Architecture Co-Design for Embodied AI-powered Robotic Manipulation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915332243849216 |
|---|---|
| author | Huang, Yiyang Hao, Yuhui Yu, Bo Yan, Feng Yang, Yuxin Min, Feng Han, Yinhe Ma, Lin Liu, Shaoshan Liu, Qiang Gan, Yiming |
| author_facet | Huang, Yiyang Hao, Yuhui Yu, Bo Yan, Feng Yang, Yuxin Min, Feng Han, Yinhe Ma, Lin Liu, Shaoshan Liu, Qiang Gan, Yiming |
| contents | Embodied AI robots have the potential to fundamentally improve the way human beings live and manufacture. Continued progress in the burgeoning field of using large language models to control robots depends critically on an efficient computing substrate, and this trend is strongly evident in manipulation tasks. In particular, today's computing systems for embodied AI robots for manipulation tasks are designed purely based on the interest of algorithm developers, where robot actions are divided into a discrete frame basis. Such an execution pipeline creates high latency and energy consumption. This paper proposes \textsc{Corki}\xspace, an algorithm-architecture co-design framework for real-time embodied AI-powered robotic manipulation applications. We aim to decouple LLM inference, robotic control, and data communication in the embodied AI robots' compute pipeline. Instead of predicting action for one single frame, \textsc{Corki}\xspace predicts the trajectory for the near future to reduce the frequency of LLM inference. The algorithm is coupled with a hardware that accelerates transforming trajectory into actual torque signals used to control robots and an execution pipeline that parallels data communication with computation. \textsc{Corki}\xspace largely reduces LLM inference frequency by up to $5.1\times$, resulting in up to $5.9\times$ speed up. The success rate improvement can be up to 13.9\%. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_04292 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | DaDu-Corki: Algorithm-Architecture Co-Design for Embodied AI-powered Robotic Manipulation Huang, Yiyang Hao, Yuhui Yu, Bo Yan, Feng Yang, Yuxin Min, Feng Han, Yinhe Ma, Lin Liu, Shaoshan Liu, Qiang Gan, Yiming Hardware Architecture Robotics Embodied AI robots have the potential to fundamentally improve the way human beings live and manufacture. Continued progress in the burgeoning field of using large language models to control robots depends critically on an efficient computing substrate, and this trend is strongly evident in manipulation tasks. In particular, today's computing systems for embodied AI robots for manipulation tasks are designed purely based on the interest of algorithm developers, where robot actions are divided into a discrete frame basis. Such an execution pipeline creates high latency and energy consumption. This paper proposes \textsc{Corki}\xspace, an algorithm-architecture co-design framework for real-time embodied AI-powered robotic manipulation applications. We aim to decouple LLM inference, robotic control, and data communication in the embodied AI robots' compute pipeline. Instead of predicting action for one single frame, \textsc{Corki}\xspace predicts the trajectory for the near future to reduce the frequency of LLM inference. The algorithm is coupled with a hardware that accelerates transforming trajectory into actual torque signals used to control robots and an execution pipeline that parallels data communication with computation. \textsc{Corki}\xspace largely reduces LLM inference frequency by up to $5.1\times$, resulting in up to $5.9\times$ speed up. The success rate improvement can be up to 13.9\%. |
| title | DaDu-Corki: Algorithm-Architecture Co-Design for Embodied AI-powered Robotic Manipulation |
| topic | Hardware Architecture Robotics |
| url | https://arxiv.org/abs/2407.04292 |