PVI: Plug-in Visual Injection for Vision-Language-Action Models
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910051693756416 |
|---|---|
| author | Zhang, Zezhou Zhang, Songxin Xiong, Xiao Zhang, Junjie Xie, Zejian Xi, Jingyi Mao, Zunyao Mao, Zan Mai, Zhixin Song, Zhuoyang Zhang, Jiaxing |
| author_facet | Zhang, Zezhou Zhang, Songxin Xiong, Xiao Zhang, Junjie Xie, Zejian Xi, Jingyi Mao, Zunyao Mao, Zan Mai, Zhixin Song, Zhuoyang Zhang, Jiaxing |
| contents | VLA architectures that pair a pretrained VLM with a flow-matching action expert have emerged as a strong paradigm for language-conditioned manipulation. Yet the VLM, optimized for semantic abstraction and typically conditioned on static visual observations, tends to attenuate fine-grained geometric cues and often lacks explicit temporal evidence for the action expert. Prior work mitigates this by injecting auxiliary visual features, but existing approaches either focus on static spatial representations or require substantial architectural modifications to accommodate temporal inputs, leaving temporal information underexplored. We propose Plug-in Visual Injection (PVI), a lightweight, encoder-agnostic module that attaches to a pretrained action expert and injects auxiliary visual representations via zero-initialized residual pathways, preserving pretrained behavior with only single-stage fine-tuning. Using PVI, we obtain consistent gains over the base policy and a range of competitive alternative injection strategies, and our controlled study shows that temporal video features (V-JEPA2) outperform strong static image features (DINOv2), with the largest gains on multi-phase tasks requiring state tracking and coordination. Real-robot experiments on long-horizon bimanual cloth folding further demonstrate the practicality of PVI beyond simulation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_12772 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | PVI: Plug-in Visual Injection for Vision-Language-Action Models Zhang, Zezhou Zhang, Songxin Xiong, Xiao Zhang, Junjie Xie, Zejian Xi, Jingyi Mao, Zunyao Mao, Zan Mai, Zhixin Song, Zhuoyang Zhang, Jiaxing Computer Vision and Pattern Recognition Machine Learning Robotics VLA architectures that pair a pretrained VLM with a flow-matching action expert have emerged as a strong paradigm for language-conditioned manipulation. Yet the VLM, optimized for semantic abstraction and typically conditioned on static visual observations, tends to attenuate fine-grained geometric cues and often lacks explicit temporal evidence for the action expert. Prior work mitigates this by injecting auxiliary visual features, but existing approaches either focus on static spatial representations or require substantial architectural modifications to accommodate temporal inputs, leaving temporal information underexplored. We propose Plug-in Visual Injection (PVI), a lightweight, encoder-agnostic module that attaches to a pretrained action expert and injects auxiliary visual representations via zero-initialized residual pathways, preserving pretrained behavior with only single-stage fine-tuning. Using PVI, we obtain consistent gains over the base policy and a range of competitive alternative injection strategies, and our controlled study shows that temporal video features (V-JEPA2) outperform strong static image features (DINOv2), with the largest gains on multi-phase tasks requiring state tracking and coordination. Real-robot experiments on long-horizon bimanual cloth folding further demonstrate the practicality of PVI beyond simulation. |
| title | PVI: Plug-in Visual Injection for Vision-Language-Action Models |
| topic | Computer Vision and Pattern Recognition Machine Learning Robotics |
| url | https://arxiv.org/abs/2603.12772 |