Addressing Failures in Robotics using Vision-Based Language Models (VLMs) and Behavior Trees (BT)
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916466117312512 |
|---|---|
| author | Ahmad, Faseeh Styrud, Jonathan Krueger, Volker |
| author_facet | Ahmad, Faseeh Styrud, Jonathan Krueger, Volker |
| contents | In this paper, we propose an approach that combines Vision Language Models (VLMs) and Behavior Trees (BTs) to address failures in robotics. Current robotic systems can handle known failures with pre-existing recovery strategies, but they are often ill-equipped to manage unknown failures or anomalies. We introduce VLMs as a monitoring tool to detect and identify failures during task execution. Additionally, VLMs generate missing conditions or skill templates that are then incorporated into the BT, ensuring the system can autonomously address similar failures in future tasks. We validate our approach through simulations in several failure scenarios. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_01568 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Addressing Failures in Robotics using Vision-Based Language Models (VLMs) and Behavior Trees (BT) Ahmad, Faseeh Styrud, Jonathan Krueger, Volker Robotics In this paper, we propose an approach that combines Vision Language Models (VLMs) and Behavior Trees (BTs) to address failures in robotics. Current robotic systems can handle known failures with pre-existing recovery strategies, but they are often ill-equipped to manage unknown failures or anomalies. We introduce VLMs as a monitoring tool to detect and identify failures during task execution. Additionally, VLMs generate missing conditions or skill templates that are then incorporated into the BT, ensuring the system can autonomously address similar failures in future tasks. We validate our approach through simulations in several failure scenarios. |
| title | Addressing Failures in Robotics using Vision-Based Language Models (VLMs) and Behavior Trees (BT) |
| topic | Robotics |
| url | https://arxiv.org/abs/2411.01568 |