CARTIER: Cartographic lAnguage Reasoning Targeted at Instruction Execution for Robots
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909090191507456 |
|---|---|
| author | Rivkin, Dmitriy Kakodkar, Nikhil Hogan, Francois Baghi, Bobak H. Dudek, Gregory |
| author_facet | Rivkin, Dmitriy Kakodkar, Nikhil Hogan, Francois Baghi, Bobak H. Dudek, Gregory |
| contents | This work explores the capacity of large language models (LLMs) to address problems at the intersection of spatial planning and natural language interfaces for navigation. We focus on following complex instructions that are more akin to natural conversation than traditional explicit procedural directives typically seen in robotics. Unlike most prior work where navigation directives are provided as simple imperative commands (e.g., "go to the fridge"), we examine implicit directives obtained through conversational interactions.We leverage the 3D simulator AI2Thor to create household query scenarios at scale, and augment it by adding complex language queries for 40 object types. We demonstrate that a robot using our method CARTIER (Cartographic lAnguage Reasoning Targeted at Instruction Execution for Robots) can parse descriptive language queries up to 42% more reliably than existing LLM-enabled methods by exploiting the ability of LLMs to interpret the user interaction in the context of the objects in the scenario. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2307_11865 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | CARTIER: Cartographic lAnguage Reasoning Targeted at Instruction Execution for Robots Rivkin, Dmitriy Kakodkar, Nikhil Hogan, Francois Baghi, Bobak H. Dudek, Gregory Robotics Computation and Language This work explores the capacity of large language models (LLMs) to address problems at the intersection of spatial planning and natural language interfaces for navigation. We focus on following complex instructions that are more akin to natural conversation than traditional explicit procedural directives typically seen in robotics. Unlike most prior work where navigation directives are provided as simple imperative commands (e.g., "go to the fridge"), we examine implicit directives obtained through conversational interactions.We leverage the 3D simulator AI2Thor to create household query scenarios at scale, and augment it by adding complex language queries for 40 object types. We demonstrate that a robot using our method CARTIER (Cartographic lAnguage Reasoning Targeted at Instruction Execution for Robots) can parse descriptive language queries up to 42% more reliably than existing LLM-enabled methods by exploiting the ability of LLMs to interpret the user interaction in the context of the objects in the scenario. |
| title | CARTIER: Cartographic lAnguage Reasoning Targeted at Instruction Execution for Robots |
| topic | Robotics Computation and Language |
| url | https://arxiv.org/abs/2307.11865 |