Adapt2Reward: Adapting Video-Language Models to Generalizable Robotic Rewards via Failure Prompts
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913439259033600 |
|---|---|
| author | Yang, Yanting Chen, Minghao Qiu, Qibo Wu, Jiahao Wang, Wenxiao Lin, Binbin Guan, Ziyu He, Xiaofei |
| author_facet | Yang, Yanting Chen, Minghao Qiu, Qibo Wu, Jiahao Wang, Wenxiao Lin, Binbin Guan, Ziyu He, Xiaofei |
| contents | For a general-purpose robot to operate in reality, executing a broad range of instructions across various environments is imperative. Central to the reinforcement learning and planning for such robotic agents is a generalizable reward function. Recent advances in vision-language models, such as CLIP, have shown remarkable performance in the domain of deep learning, paving the way for open-domain visual recognition. However, collecting data on robots executing various language instructions across multiple environments remains a challenge. This paper aims to transfer video-language models with robust generalization into a generalizable language-conditioned reward function, only utilizing robot video data from a minimal amount of tasks in a singular environment. Unlike common robotic datasets used for training reward functions, human video-language datasets rarely contain trivial failure videos. To enhance the model's ability to distinguish between successful and failed robot executions, we cluster failure video features to enable the model to identify patterns within. For each cluster, we integrate a newly trained failure prompt into the text encoder to represent the corresponding failure mode. Our language-conditioned reward function shows outstanding generalization to new environments and new instructions for robot planning and reinforcement learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_14872 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Adapt2Reward: Adapting Video-Language Models to Generalizable Robotic Rewards via Failure Prompts Yang, Yanting Chen, Minghao Qiu, Qibo Wu, Jiahao Wang, Wenxiao Lin, Binbin Guan, Ziyu He, Xiaofei Computer Vision and Pattern Recognition Robotics For a general-purpose robot to operate in reality, executing a broad range of instructions across various environments is imperative. Central to the reinforcement learning and planning for such robotic agents is a generalizable reward function. Recent advances in vision-language models, such as CLIP, have shown remarkable performance in the domain of deep learning, paving the way for open-domain visual recognition. However, collecting data on robots executing various language instructions across multiple environments remains a challenge. This paper aims to transfer video-language models with robust generalization into a generalizable language-conditioned reward function, only utilizing robot video data from a minimal amount of tasks in a singular environment. Unlike common robotic datasets used for training reward functions, human video-language datasets rarely contain trivial failure videos. To enhance the model's ability to distinguish between successful and failed robot executions, we cluster failure video features to enable the model to identify patterns within. For each cluster, we integrate a newly trained failure prompt into the text encoder to represent the corresponding failure mode. Our language-conditioned reward function shows outstanding generalization to new environments and new instructions for robot planning and reinforcement learning. |
| title | Adapt2Reward: Adapting Video-Language Models to Generalizable Robotic Rewards via Failure Prompts |
| topic | Computer Vision and Pattern Recognition Robotics |
| url | https://arxiv.org/abs/2407.14872 |