Steerable Vision-Language-Action Policies for Embodied Reasoning and Hierarchical Control
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917384957198336 |
|---|---|
| author | Chen, William Bhatia, Jagdeep Singh Glossop, Catherine Mathihalli, Nikhil Doshi, Ria Tang, Andy Driess, Danny Pertsch, Karl Levine, Sergey |
| author_facet | Chen, William Bhatia, Jagdeep Singh Glossop, Catherine Mathihalli, Nikhil Doshi, Ria Tang, Andy Driess, Danny Pertsch, Karl Levine, Sergey |
| contents | Pretrained vision-language models (VLMs) can make semantic and visual inferences across diverse settings, providing valuable common-sense priors for robotic control. However, effectively grounding this knowledge in robot behaviors remains an open challenge. Prior methods often employ a hierarchical approach where VLMs reason over high-level commands to be executed by separate low-level policies, e.g., vision-language-action models (VLAs). The interface between VLMs and VLAs is usually natural language task instructions, which fundamentally limits how much VLM reasoning can steer low-level behavior. We thus introduce Steerable Policies: VLAs trained on rich synthetic commands at various levels of abstraction, like subtasks, motions, and grounded pixel coordinates. By improving low-level controllability, Steerable Policies can unlock pretrained knowledge in VLMs, enabling improved task generalization. We demonstrate this benefit by controlling our Steerable Policies with both a learned high-level embodied reasoner and an off-the-shelf VLM prompted to reason over command abstractions via in-context learning. Across extensive real-world manipulation experiments, these two novel methods outperform prior embodied reasoning VLAs and VLM-based hierarchical baselines, including on challenging generalization and long-horizon tasks.
Website: steerable-policies.github.io |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_13193 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Steerable Vision-Language-Action Policies for Embodied Reasoning and Hierarchical Control Chen, William Bhatia, Jagdeep Singh Glossop, Catherine Mathihalli, Nikhil Doshi, Ria Tang, Andy Driess, Danny Pertsch, Karl Levine, Sergey Robotics Pretrained vision-language models (VLMs) can make semantic and visual inferences across diverse settings, providing valuable common-sense priors for robotic control. However, effectively grounding this knowledge in robot behaviors remains an open challenge. Prior methods often employ a hierarchical approach where VLMs reason over high-level commands to be executed by separate low-level policies, e.g., vision-language-action models (VLAs). The interface between VLMs and VLAs is usually natural language task instructions, which fundamentally limits how much VLM reasoning can steer low-level behavior. We thus introduce Steerable Policies: VLAs trained on rich synthetic commands at various levels of abstraction, like subtasks, motions, and grounded pixel coordinates. By improving low-level controllability, Steerable Policies can unlock pretrained knowledge in VLMs, enabling improved task generalization. We demonstrate this benefit by controlling our Steerable Policies with both a learned high-level embodied reasoner and an off-the-shelf VLM prompted to reason over command abstractions via in-context learning. Across extensive real-world manipulation experiments, these two novel methods outperform prior embodied reasoning VLAs and VLM-based hierarchical baselines, including on challenging generalization and long-horizon tasks. Website: steerable-policies.github.io |
| title | Steerable Vision-Language-Action Policies for Embodied Reasoning and Hierarchical Control |
| topic | Robotics |
| url | https://arxiv.org/abs/2602.13193 |