Do You Need Proprioceptive States in Visuomotor Policies?
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915510470311936 |
|---|---|
| author | Zhao, Juntu Lu, Wenbo Zhang, Di Liu, Yufeng Liang, Yushen Zhang, Tianluo Cao, Yifeng Xie, Junyuan Hu, Yingdong Wang, Shengjie Guo, Junliang Wang, Dequan Gao, Yang |
| author_facet | Zhao, Juntu Lu, Wenbo Zhang, Di Liu, Yufeng Liang, Yushen Zhang, Tianluo Cao, Yifeng Xie, Junyuan Hu, Yingdong Wang, Shengjie Guo, Junliang Wang, Dequan Gao, Yang |
| contents | Imitation-learning-based visuomotor policies have been widely used in robot manipulation, where both visual observations and proprioceptive states are typically adopted together for precise control. However, in this study, we find that this common practice makes the policy overly reliant on the proprioceptive state input, which causes overfitting to the training trajectories and results in poor spatial generalization. On the contrary, we propose the State-free Policy, removing the proprioceptive state input and predicting actions only conditioned on visual observations. The State-free Policy is built in the relative end-effector action space, and should ensure the full task-relevant visual observations, here provided by dual wide-angle wrist cameras. Empirical results demonstrate that the State-free policy achieves significantly stronger spatial generalization than the state-based policy: in real-world tasks such as pick-and-place, challenging shirt-folding, and complex whole-body manipulation, spanning multiple robot embodiments, the average success rate improves from 0% to 85% in height generalization and from 6% to 64% in horizontal generalization. Furthermore, they also show advantages in data efficiency and cross-embodiment adaptation, enhancing their practicality for real-world deployment. Discover more by visiting: https://statefreepolicy.github.io. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_18644 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Do You Need Proprioceptive States in Visuomotor Policies? Zhao, Juntu Lu, Wenbo Zhang, Di Liu, Yufeng Liang, Yushen Zhang, Tianluo Cao, Yifeng Xie, Junyuan Hu, Yingdong Wang, Shengjie Guo, Junliang Wang, Dequan Gao, Yang Robotics Artificial Intelligence Imitation-learning-based visuomotor policies have been widely used in robot manipulation, where both visual observations and proprioceptive states are typically adopted together for precise control. However, in this study, we find that this common practice makes the policy overly reliant on the proprioceptive state input, which causes overfitting to the training trajectories and results in poor spatial generalization. On the contrary, we propose the State-free Policy, removing the proprioceptive state input and predicting actions only conditioned on visual observations. The State-free Policy is built in the relative end-effector action space, and should ensure the full task-relevant visual observations, here provided by dual wide-angle wrist cameras. Empirical results demonstrate that the State-free policy achieves significantly stronger spatial generalization than the state-based policy: in real-world tasks such as pick-and-place, challenging shirt-folding, and complex whole-body manipulation, spanning multiple robot embodiments, the average success rate improves from 0% to 85% in height generalization and from 6% to 64% in horizontal generalization. Furthermore, they also show advantages in data efficiency and cross-embodiment adaptation, enhancing their practicality for real-world deployment. Discover more by visiting: https://statefreepolicy.github.io. |
| title | Do You Need Proprioceptive States in Visuomotor Policies? |
| topic | Robotics Artificial Intelligence |
| url | https://arxiv.org/abs/2509.18644 |