Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2309.09182 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913191682899968 |
|---|---|
| author | Dai, Zhirui Asgharivaskasi, Arash Duong, Thai Lin, Shusen Tzes, Maria-Elizabeth Pappas, George Atanasov, Nikolay |
| author_facet | Dai, Zhirui Asgharivaskasi, Arash Duong, Thai Lin, Shusen Tzes, Maria-Elizabeth Pappas, George Atanasov, Nikolay |
| contents | Recent advances in metric, semantic, and topological mapping have equipped autonomous robots with semantic concept grounding capabilities to interpret natural language tasks. This work aims to leverage these new capabilities with an efficient task planning algorithm for hierarchical metric-semantic models. We consider a scene graph representation of the environment and utilize a large language model (LLM) to convert a natural language task into a linear temporal logic (LTL) automaton. Our main contribution is to enable optimal hierarchical LTL planning with LLM guidance over scene graphs. To achieve efficiency, we construct a hierarchical planning domain that captures the attributes and connectivity of the scene graph and the task automaton, and provide semantic guidance via an LLM heuristic function. To guarantee optimality, we design an LTL heuristic function that is provably consistent and supplements the potentially inadmissible LLM guidance in multi-heuristic planning. We demonstrate efficient planning of complex natural language tasks in scene graphs of virtualized real environments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_09182 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Optimal Scene Graph Planning with Large Language Model Guidance Dai, Zhirui Asgharivaskasi, Arash Duong, Thai Lin, Shusen Tzes, Maria-Elizabeth Pappas, George Atanasov, Nikolay Robotics Recent advances in metric, semantic, and topological mapping have equipped autonomous robots with semantic concept grounding capabilities to interpret natural language tasks. This work aims to leverage these new capabilities with an efficient task planning algorithm for hierarchical metric-semantic models. We consider a scene graph representation of the environment and utilize a large language model (LLM) to convert a natural language task into a linear temporal logic (LTL) automaton. Our main contribution is to enable optimal hierarchical LTL planning with LLM guidance over scene graphs. To achieve efficiency, we construct a hierarchical planning domain that captures the attributes and connectivity of the scene graph and the task automaton, and provide semantic guidance via an LLM heuristic function. To guarantee optimality, we design an LTL heuristic function that is provably consistent and supplements the potentially inadmissible LLM guidance in multi-heuristic planning. We demonstrate efficient planning of complex natural language tasks in scene graphs of virtualized real environments. |
| title | Optimal Scene Graph Planning with Large Language Model Guidance |
| topic | Robotics |
| url | https://arxiv.org/abs/2309.09182 |