Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2411.04549 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909379686563840 |
|---|---|
| author | Ma, Yecheng Jason Hejna, Joey Wahid, Ayzaan Fu, Chuyuan Shah, Dhruv Liang, Jacky Xu, Zhuo Kirmani, Sean Xu, Peng Driess, Danny Xiao, Ted Tompson, Jonathan Bastani, Osbert Jayaraman, Dinesh Yu, Wenhao Zhang, Tingnan Sadigh, Dorsa Xia, Fei |
| author_facet | Ma, Yecheng Jason Hejna, Joey Wahid, Ayzaan Fu, Chuyuan Shah, Dhruv Liang, Jacky Xu, Zhuo Kirmani, Sean Xu, Peng Driess, Danny Xiao, Ted Tompson, Jonathan Bastani, Osbert Jayaraman, Dinesh Yu, Wenhao Zhang, Tingnan Sadigh, Dorsa Xia, Fei |
| contents | Predicting temporal progress from visual trajectories is important for intelligent robots that can learn, adapt, and improve. However, learning such progress estimator, or temporal value function, across different tasks and domains requires both a large amount of diverse data and methods which can scale and generalize. To address these challenges, we present Generative Value Learning (\GVL), a universal value function estimator that leverages the world knowledge embedded in vision-language models (VLMs) to predict task progress. Naively asking a VLM to predict values for a video sequence performs poorly due to the strong temporal correlation between successive frames. Instead, GVL poses value estimation as a temporal ordering problem over shuffled video frames; this seemingly more challenging task encourages VLMs to more fully exploit their underlying semantic and temporal grounding capabilities to differentiate frames based on their perceived task progress, consequently producing significantly better value predictions. Without any robot or task specific training, GVL can in-context zero-shot and few-shot predict effective values for more than 300 distinct real-world tasks across diverse robot platforms, including challenging bimanual manipulation tasks. Furthermore, we demonstrate that GVL permits flexible multi-modal in-context learning via examples from heterogeneous tasks and embodiments, such as human videos. The generality of GVL enables various downstream applications pertinent to visuomotor policy learning, including dataset filtering, success detection, and advantage-weighted regression -- all without any model training or finetuning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_04549 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Vision Language Models are In-Context Value Learners Ma, Yecheng Jason Hejna, Joey Wahid, Ayzaan Fu, Chuyuan Shah, Dhruv Liang, Jacky Xu, Zhuo Kirmani, Sean Xu, Peng Driess, Danny Xiao, Ted Tompson, Jonathan Bastani, Osbert Jayaraman, Dinesh Yu, Wenhao Zhang, Tingnan Sadigh, Dorsa Xia, Fei Robotics Artificial Intelligence Machine Learning Predicting temporal progress from visual trajectories is important for intelligent robots that can learn, adapt, and improve. However, learning such progress estimator, or temporal value function, across different tasks and domains requires both a large amount of diverse data and methods which can scale and generalize. To address these challenges, we present Generative Value Learning (\GVL), a universal value function estimator that leverages the world knowledge embedded in vision-language models (VLMs) to predict task progress. Naively asking a VLM to predict values for a video sequence performs poorly due to the strong temporal correlation between successive frames. Instead, GVL poses value estimation as a temporal ordering problem over shuffled video frames; this seemingly more challenging task encourages VLMs to more fully exploit their underlying semantic and temporal grounding capabilities to differentiate frames based on their perceived task progress, consequently producing significantly better value predictions. Without any robot or task specific training, GVL can in-context zero-shot and few-shot predict effective values for more than 300 distinct real-world tasks across diverse robot platforms, including challenging bimanual manipulation tasks. Furthermore, we demonstrate that GVL permits flexible multi-modal in-context learning via examples from heterogeneous tasks and embodiments, such as human videos. The generality of GVL enables various downstream applications pertinent to visuomotor policy learning, including dataset filtering, success detection, and advantage-weighted regression -- all without any model training or finetuning. |
| title | Vision Language Models are In-Context Value Learners |
| topic | Robotics Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2411.04549 |