Salvato in:
| Autori principali: | , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2509.15212 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914046252417024 |
|---|---|
| author | Jiang, Yuming Huang, Siteng Xue, Shengke Zhao, Yaxi Cen, Jun Leng, Sicong Li, Kehan Guo, Jiayan Wang, Kexiang Chen, Mingxiu Wang, Fan Zhao, Deli Li, Xin |
| author_facet | Jiang, Yuming Huang, Siteng Xue, Shengke Zhao, Yaxi Cen, Jun Leng, Sicong Li, Kehan Guo, Jiayan Wang, Kexiang Chen, Mingxiu Wang, Fan Zhao, Deli Li, Xin |
| contents | This paper presents RynnVLA-001, a vision-language-action(VLA) model built upon large-scale video generative pretraining from human demonstrations. We propose a novel two-stage pretraining methodology. The first stage, Ego-Centric Video Generative Pretraining, trains an Image-to-Video model on 12M ego-centric manipulation videos to predict future frames conditioned on an initial frame and a language instruction. The second stage, Human-Centric Trajectory-Aware Modeling, extends this by jointly predicting future keypoint trajectories, thereby effectively bridging visual frame prediction with action prediction. Furthermore, to enhance action representation, we propose ActionVAE, a variational autoencoder that compresses sequences of actions into compact latent embeddings, reducing the complexity of the VLA output space. When finetuned on the same downstream robotics datasets, RynnVLA-001 achieves superior performance over state-of-the-art baselines, demonstrating that the proposed pretraining strategy provides a more effective initialization for VLA models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_15212 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | RynnVLA-001: Using Human Demonstrations to Improve Robot Manipulation Jiang, Yuming Huang, Siteng Xue, Shengke Zhao, Yaxi Cen, Jun Leng, Sicong Li, Kehan Guo, Jiayan Wang, Kexiang Chen, Mingxiu Wang, Fan Zhao, Deli Li, Xin Computer Vision and Pattern Recognition Robotics This paper presents RynnVLA-001, a vision-language-action(VLA) model built upon large-scale video generative pretraining from human demonstrations. We propose a novel two-stage pretraining methodology. The first stage, Ego-Centric Video Generative Pretraining, trains an Image-to-Video model on 12M ego-centric manipulation videos to predict future frames conditioned on an initial frame and a language instruction. The second stage, Human-Centric Trajectory-Aware Modeling, extends this by jointly predicting future keypoint trajectories, thereby effectively bridging visual frame prediction with action prediction. Furthermore, to enhance action representation, we propose ActionVAE, a variational autoencoder that compresses sequences of actions into compact latent embeddings, reducing the complexity of the VLA output space. When finetuned on the same downstream robotics datasets, RynnVLA-001 achieves superior performance over state-of-the-art baselines, demonstrating that the proposed pretraining strategy provides a more effective initialization for VLA models. |
| title | RynnVLA-001: Using Human Demonstrations to Improve Robot Manipulation |
| topic | Computer Vision and Pattern Recognition Robotics |
| url | https://arxiv.org/abs/2509.15212 |