villa-X: Enhancing Latent Action Modeling in Vision-Language-Action Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912603994849280 |
|---|---|
| author | Chen, Xiaoyu Wei, Hangxing Zhang, Pushi Zhang, Chuheng Wang, Kaixin Guo, Yanjiang Yang, Rushuai Wang, Yucen Xiao, Xinquan Zhao, Li Chen, Jianyu Bian, Jiang |
| author_facet | Chen, Xiaoyu Wei, Hangxing Zhang, Pushi Zhang, Chuheng Wang, Kaixin Guo, Yanjiang Yang, Rushuai Wang, Yucen Xiao, Xinquan Zhao, Li Chen, Jianyu Bian, Jiang |
| contents | Vision-Language-Action (VLA) models have emerged as a popular paradigm for learning robot manipulation policies that can follow language instructions and generalize to novel scenarios. Recent works have begun to explore the incorporation of latent actions, abstract representations of motion between two frames, into VLA pre-training. In this paper, we introduce villa-X, a novel Vision-Language-Latent-Action (ViLLA) framework that advances latent action modeling for learning generalizable robot manipulation policies. Our approach improves both how latent actions are learned and how they are incorporated into VLA pre-training. We demonstrate that villa-X can generate latent action plans in a zero-shot fashion, even for unseen embodiments and open-vocabulary symbolic understanding. This capability enables villa-X to achieve superior performance across diverse simulation tasks in SIMPLER and on two real-world robotic setups involving both gripper and dexterous hand manipulation. These results establish villa-X as a principled and scalable paradigm for learning generalizable robot manipulation policies. We believe it provides a strong foundation for future research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_23682 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | villa-X: Enhancing Latent Action Modeling in Vision-Language-Action Models Chen, Xiaoyu Wei, Hangxing Zhang, Pushi Zhang, Chuheng Wang, Kaixin Guo, Yanjiang Yang, Rushuai Wang, Yucen Xiao, Xinquan Zhao, Li Chen, Jianyu Bian, Jiang Robotics Artificial Intelligence Machine Learning Vision-Language-Action (VLA) models have emerged as a popular paradigm for learning robot manipulation policies that can follow language instructions and generalize to novel scenarios. Recent works have begun to explore the incorporation of latent actions, abstract representations of motion between two frames, into VLA pre-training. In this paper, we introduce villa-X, a novel Vision-Language-Latent-Action (ViLLA) framework that advances latent action modeling for learning generalizable robot manipulation policies. Our approach improves both how latent actions are learned and how they are incorporated into VLA pre-training. We demonstrate that villa-X can generate latent action plans in a zero-shot fashion, even for unseen embodiments and open-vocabulary symbolic understanding. This capability enables villa-X to achieve superior performance across diverse simulation tasks in SIMPLER and on two real-world robotic setups involving both gripper and dexterous hand manipulation. These results establish villa-X as a principled and scalable paradigm for learning generalizable robot manipulation policies. We believe it provides a strong foundation for future research. |
| title | villa-X: Enhancing Latent Action Modeling in Vision-Language-Action Models |
| topic | Robotics Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2507.23682 |