Socratic Planner: Self-QA-Based Zero-Shot Planning for Embodied Instruction Following
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915213445431296 |
|---|---|
| author | Shin, Suyeon jeon, Sujin Kim, Junghyun Kang, Gi-Cheon Zhang, Byoung-Tak |
| author_facet | Shin, Suyeon jeon, Sujin Kim, Junghyun Kang, Gi-Cheon Zhang, Byoung-Tak |
| contents | Embodied Instruction Following (EIF) is the task of executing natural language instructions by navigating and interacting with objects in interactive environments. A key challenge in EIF is compositional task planning, typically addressed through supervised learning or few-shot in-context learning with labeled data. To this end, we introduce the Socratic Planner, a self-QA-based zero-shot planning method that infers an appropriate plan without any further training. The Socratic Planner first facilitates self-questioning and answering by the Large Language Model (LLM), which in turn helps generate a sequence of subgoals. While executing the subgoals, an embodied agent may encounter unexpected situations, such as unforeseen obstacles. The Socratic Planner then adjusts plans based on dense visual feedback through a visually-grounded re-planning mechanism. Experiments demonstrate the effectiveness of the Socratic Planner, outperforming current state-of-the-art planning models on the ALFRED benchmark across all metrics, particularly excelling in long-horizon tasks that demand complex inference. We further demonstrate its real-world applicability through deployment on a physical robot for long-horizon tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_15190 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Socratic Planner: Self-QA-Based Zero-Shot Planning for Embodied Instruction Following Shin, Suyeon jeon, Sujin Kim, Junghyun Kang, Gi-Cheon Zhang, Byoung-Tak Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition Robotics 68T01 (Primary) 68T40, 68T50, 68T45 (Secondary) Embodied Instruction Following (EIF) is the task of executing natural language instructions by navigating and interacting with objects in interactive environments. A key challenge in EIF is compositional task planning, typically addressed through supervised learning or few-shot in-context learning with labeled data. To this end, we introduce the Socratic Planner, a self-QA-based zero-shot planning method that infers an appropriate plan without any further training. The Socratic Planner first facilitates self-questioning and answering by the Large Language Model (LLM), which in turn helps generate a sequence of subgoals. While executing the subgoals, an embodied agent may encounter unexpected situations, such as unforeseen obstacles. The Socratic Planner then adjusts plans based on dense visual feedback through a visually-grounded re-planning mechanism. Experiments demonstrate the effectiveness of the Socratic Planner, outperforming current state-of-the-art planning models on the ALFRED benchmark across all metrics, particularly excelling in long-horizon tasks that demand complex inference. We further demonstrate its real-world applicability through deployment on a physical robot for long-horizon tasks. |
| title | Socratic Planner: Self-QA-Based Zero-Shot Planning for Embodied Instruction Following |
| topic | Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition Robotics 68T01 (Primary) 68T40, 68T50, 68T45 (Secondary) |
| url | https://arxiv.org/abs/2404.15190 |