See Once, Then Act: Vision-Language-Action Model with Task Learning from One-Shot Video Demonstrations
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911308244320256 |
|---|---|
| author | Chen, Guangyan Wang, Meiling Shao, Qi Zhou, Zichen Mao, Weixin Cui, Te Zhu, Minzhao Deng, Yinan Yang, Luojie Zhang, Zhanqi Yang, Yi Chen, Hua Yue, Yufeng |
| author_facet | Chen, Guangyan Wang, Meiling Shao, Qi Zhou, Zichen Mao, Weixin Cui, Te Zhu, Minzhao Deng, Yinan Yang, Luojie Zhang, Zhanqi Yang, Yi Chen, Hua Yue, Yufeng |
| contents | Developing robust and general-purpose manipulation policies represents a fundamental objective in robotics research. While Vision-Language-Action (VLA) models have demonstrated promising capabilities for end-to-end robot control, existing approaches still exhibit limited generalization to tasks beyond their training distributions. In contrast, humans possess remarkable proficiency in acquiring novel skills by simply observing others performing them once. Inspired by this capability, we propose ViVLA, a generalist robotic manipulation policy that achieves efficient task learning from a single expert demonstration video at test time. Our approach jointly processes an expert demonstration video alongside the robot's visual observations to predict both the demonstrated action sequences and subsequent robot actions, effectively distilling fine-grained manipulation knowledge from expert behavior and transferring it seamlessly to the agent. To enhance the performance of ViVLA, we develop a scalable expert-agent pair data generation pipeline capable of synthesizing paired trajectories from easily accessible human videos, further augmented by curated pairs from publicly available datasets. This pipeline produces a total of 892,911 expert-agent samples for training ViVLA. Experimental results demonstrate that our ViVLA is able to acquire novel manipulation skills from only a single expert demonstration video at test time. Our approach achieves over 30% improvement on unseen LIBERO tasks and maintains above 35% gains with cross-embodiment videos. Real-world experiments demonstrate effective learning from human videos, yielding more than 38% improvement on unseen tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_07582 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | See Once, Then Act: Vision-Language-Action Model with Task Learning from One-Shot Video Demonstrations Chen, Guangyan Wang, Meiling Shao, Qi Zhou, Zichen Mao, Weixin Cui, Te Zhu, Minzhao Deng, Yinan Yang, Luojie Zhang, Zhanqi Yang, Yi Chen, Hua Yue, Yufeng Robotics Developing robust and general-purpose manipulation policies represents a fundamental objective in robotics research. While Vision-Language-Action (VLA) models have demonstrated promising capabilities for end-to-end robot control, existing approaches still exhibit limited generalization to tasks beyond their training distributions. In contrast, humans possess remarkable proficiency in acquiring novel skills by simply observing others performing them once. Inspired by this capability, we propose ViVLA, a generalist robotic manipulation policy that achieves efficient task learning from a single expert demonstration video at test time. Our approach jointly processes an expert demonstration video alongside the robot's visual observations to predict both the demonstrated action sequences and subsequent robot actions, effectively distilling fine-grained manipulation knowledge from expert behavior and transferring it seamlessly to the agent. To enhance the performance of ViVLA, we develop a scalable expert-agent pair data generation pipeline capable of synthesizing paired trajectories from easily accessible human videos, further augmented by curated pairs from publicly available datasets. This pipeline produces a total of 892,911 expert-agent samples for training ViVLA. Experimental results demonstrate that our ViVLA is able to acquire novel manipulation skills from only a single expert demonstration video at test time. Our approach achieves over 30% improvement on unseen LIBERO tasks and maintains above 35% gains with cross-embodiment videos. Real-world experiments demonstrate effective learning from human videos, yielding more than 38% improvement on unseen tasks. |
| title | See Once, Then Act: Vision-Language-Action Model with Task Learning from One-Shot Video Demonstrations |
| topic | Robotics |
| url | https://arxiv.org/abs/2512.07582 |