Real-World Cooking Robot System from Recipes Based on Food State Recognition Using Foundation Models and PDDL
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917796070293504 |
|---|---|
| author | Kanazawa, Naoaki Kawaharazuka, Kento Obinata, Yoshiki Okada, Kei Inaba, Masayuki |
| author_facet | Kanazawa, Naoaki Kawaharazuka, Kento Obinata, Yoshiki Okada, Kei Inaba, Masayuki |
| contents | Although there is a growing demand for cooking behaviours as one of the expected tasks for robots, a series of cooking behaviours based on new recipe descriptions by robots in the real world has not yet been realised. In this study, we propose a robot system that integrates real-world executable robot cooking behaviour planning using the Large Language Model (LLM) and classical planning of PDDL descriptions, and food ingredient state recognition learning from a small number of data using the Vision-Language model (VLM). We succeeded in experiments in which PR2, a dual-armed wheeled robot, performed cooking from arranged new recipes in a real-world environment, and confirmed the effectiveness of the proposed system. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_02874 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Real-World Cooking Robot System from Recipes Based on Food State Recognition Using Foundation Models and PDDL Kanazawa, Naoaki Kawaharazuka, Kento Obinata, Yoshiki Okada, Kei Inaba, Masayuki Robotics Artificial Intelligence Although there is a growing demand for cooking behaviours as one of the expected tasks for robots, a series of cooking behaviours based on new recipe descriptions by robots in the real world has not yet been realised. In this study, we propose a robot system that integrates real-world executable robot cooking behaviour planning using the Large Language Model (LLM) and classical planning of PDDL descriptions, and food ingredient state recognition learning from a small number of data using the Vision-Language model (VLM). We succeeded in experiments in which PR2, a dual-armed wheeled robot, performed cooking from arranged new recipes in a real-world environment, and confirmed the effectiveness of the proposed system. |
| title | Real-World Cooking Robot System from Recipes Based on Food State Recognition Using Foundation Models and PDDL |
| topic | Robotics Artificial Intelligence |
| url | https://arxiv.org/abs/2410.02874 |