STEER: Flexible Robotic Manipulation via Dense Language Grounding
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916470040035328 |
|---|---|
| author | Smith, Laura Irpan, Alex Arenas, Montserrat Gonzalez Kirmani, Sean Kalashnikov, Dmitry Shah, Dhruv Xiao, Ted |
| author_facet | Smith, Laura Irpan, Alex Arenas, Montserrat Gonzalez Kirmani, Sean Kalashnikov, Dmitry Shah, Dhruv Xiao, Ted |
| contents | The complexity of the real world demands robotic systems that can intelligently adapt to unseen situations. We present STEER, a robot learning framework that bridges high-level, commonsense reasoning with precise, flexible low-level control. Our approach translates complex situational awareness into actionable low-level behavior through training language-grounded policies with dense annotation. By structuring policy training around fundamental, modular manipulation skills expressed in natural language, STEER exposes an expressive interface for humans or Vision-Language Models (VLMs) to intelligently orchestrate the robot's behavior by reasoning about the task and context. Our experiments demonstrate the skills learned via STEER can be combined to synthesize novel behaviors to adapt to new situations or perform completely new tasks without additional data collection or training. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_03409 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | STEER: Flexible Robotic Manipulation via Dense Language Grounding Smith, Laura Irpan, Alex Arenas, Montserrat Gonzalez Kirmani, Sean Kalashnikov, Dmitry Shah, Dhruv Xiao, Ted Robotics Artificial Intelligence The complexity of the real world demands robotic systems that can intelligently adapt to unseen situations. We present STEER, a robot learning framework that bridges high-level, commonsense reasoning with precise, flexible low-level control. Our approach translates complex situational awareness into actionable low-level behavior through training language-grounded policies with dense annotation. By structuring policy training around fundamental, modular manipulation skills expressed in natural language, STEER exposes an expressive interface for humans or Vision-Language Models (VLMs) to intelligently orchestrate the robot's behavior by reasoning about the task and context. Our experiments demonstrate the skills learned via STEER can be combined to synthesize novel behaviors to adapt to new situations or perform completely new tasks without additional data collection or training. |
| title | STEER: Flexible Robotic Manipulation via Dense Language Grounding |
| topic | Robotics Artificial Intelligence |
| url | https://arxiv.org/abs/2411.03409 |