Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2406.10157 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913439015763968 |
|---|---|
| author | Zhou, Hantao Ji, Tianying Sommerhalder, Lukas Goerner, Michael Hendrich, Norman Zhang, Jianwei Sun, Fuchun Xu, Huazhe |
| author_facet | Zhou, Hantao Ji, Tianying Sommerhalder, Lukas Goerner, Michael Hendrich, Norman Zhang, Jianwei Sun, Fuchun Xu, Huazhe |
| contents | Minigolf is an exemplary real-world game for examining embodied intelligence, requiring challenging spatial and kinodynamic understanding to putt the ball. Additionally, reflective reasoning is required if the feasibility of a challenge is not ensured. We introduce RoboGolf, a VLM-based framework that combines dual-camera perception with closed-loop action refinement, augmented by a reflective equilibrium loop. The core of both loops is powered by finetuned VLMs. We analyze the capabilities of the framework in an offline inference setting, relying on an extensive set of recorded trajectories. Exemplary demonstrations of the analyzed problem domain are available at https://jity16.github.io/RoboGolf/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_10157 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | RoboGolf: Mastering Real-World Minigolf with a Reflective Multi-Modality Vision-Language Model Zhou, Hantao Ji, Tianying Sommerhalder, Lukas Goerner, Michael Hendrich, Norman Zhang, Jianwei Sun, Fuchun Xu, Huazhe Robotics Artificial Intelligence Minigolf is an exemplary real-world game for examining embodied intelligence, requiring challenging spatial and kinodynamic understanding to putt the ball. Additionally, reflective reasoning is required if the feasibility of a challenge is not ensured. We introduce RoboGolf, a VLM-based framework that combines dual-camera perception with closed-loop action refinement, augmented by a reflective equilibrium loop. The core of both loops is powered by finetuned VLMs. We analyze the capabilities of the framework in an offline inference setting, relying on an extensive set of recorded trajectories. Exemplary demonstrations of the analyzed problem domain are available at https://jity16.github.io/RoboGolf/ |
| title | RoboGolf: Mastering Real-World Minigolf with a Reflective Multi-Modality Vision-Language Model |
| topic | Robotics Artificial Intelligence |
| url | https://arxiv.org/abs/2406.10157 |