Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2506.16211 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908413386031104 |
|---|---|
| author | Li, Puhao Wu, Yingying Xi, Ziheng Li, Wanlin Huang, Yuzhe Zhang, Zhiyuan Chen, Yinghan Wang, Jianan Zhu, Song-Chun Liu, Tengyu Huang, Siyuan |
| author_facet | Li, Puhao Wu, Yingying Xi, Ziheng Li, Wanlin Huang, Yuzhe Zhang, Zhiyuan Chen, Yinghan Wang, Jianan Zhu, Song-Chun Liu, Tengyu Huang, Siyuan |
| contents | Learning real-world robotic manipulation is challenging, particularly when limited demonstrations are available. Existing methods for few-shot manipulation often rely on simulation-augmented data or pre-built modules like grasping and pose estimation, which struggle with sim-to-real gaps and lack extensibility. While large-scale imitation pre-training shows promise, adapting these general-purpose policies to specific tasks in data-scarce settings remains unexplored. To achieve this, we propose ControlVLA, a novel framework that bridges pre-trained VLA models with object-centric representations via a ControlNet-style architecture for efficient fine-tuning. Specifically, to introduce object-centric conditions without overwriting prior knowledge, ControlVLA zero-initializes a set of projection layers, allowing them to gradually adapt the pre-trained manipulation policies. In real-world experiments across 6 diverse tasks, including pouring cubes and folding clothes, our method achieves a 76.7% success rate while requiring only 10-20 demonstrations -- a significant improvement over traditional approaches that require more than 100 demonstrations to achieve comparable success. Additional experiments highlight ControlVLA's extensibility to long-horizon tasks and robustness to unseen objects and backgrounds. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_16211 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ControlVLA: Few-shot Object-centric Adaptation for Pre-trained Vision-Language-Action Models Li, Puhao Wu, Yingying Xi, Ziheng Li, Wanlin Huang, Yuzhe Zhang, Zhiyuan Chen, Yinghan Wang, Jianan Zhu, Song-Chun Liu, Tengyu Huang, Siyuan Robotics Learning real-world robotic manipulation is challenging, particularly when limited demonstrations are available. Existing methods for few-shot manipulation often rely on simulation-augmented data or pre-built modules like grasping and pose estimation, which struggle with sim-to-real gaps and lack extensibility. While large-scale imitation pre-training shows promise, adapting these general-purpose policies to specific tasks in data-scarce settings remains unexplored. To achieve this, we propose ControlVLA, a novel framework that bridges pre-trained VLA models with object-centric representations via a ControlNet-style architecture for efficient fine-tuning. Specifically, to introduce object-centric conditions without overwriting prior knowledge, ControlVLA zero-initializes a set of projection layers, allowing them to gradually adapt the pre-trained manipulation policies. In real-world experiments across 6 diverse tasks, including pouring cubes and folding clothes, our method achieves a 76.7% success rate while requiring only 10-20 demonstrations -- a significant improvement over traditional approaches that require more than 100 demonstrations to achieve comparable success. Additional experiments highlight ControlVLA's extensibility to long-horizon tasks and robustness to unseen objects and backgrounds. |
| title | ControlVLA: Few-shot Object-centric Adaptation for Pre-trained Vision-Language-Action Models |
| topic | Robotics |
| url | https://arxiv.org/abs/2506.16211 |