OCRA: Object-Centric Learning with 3D and Tactile Priors for Human-to-Robot Action Transfer
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908887726161920 |
|---|---|
| author | Wang, Kuanning Fan, Ke Fu, Yuqian Lin, Siyu Luo, Hu Seita, Daniel Fu, Yanwei Jiang, Yu-Gang Xue, Xiangyang |
| author_facet | Wang, Kuanning Fan, Ke Fu, Yuqian Lin, Siyu Luo, Hu Seita, Daniel Fu, Yanwei Jiang, Yu-Gang Xue, Xiangyang |
| contents | We present OCRA, an Object-Centric framework for video-based human-to-Robot Action transfer that learns directly from human demonstration videos to enable robust manipulation. Object-centric learning emphasizes task-relevant objects and their interactions while filtering out irrelevant background, providing a natural and scalable way to teach robots. OCRA leverages multi-view RGB videos, the state-of-the-art 3D foundation model VGGT, and advanced detection and segmentation models to reconstruct object-centric 3D point clouds, capturing rich interactions between objects. To handle properties not easily perceived by vision alone, we incorporate tactile priors via a large-scale dataset of over one million tactile images. These 3D and tactile priors are fused through a multimodal module (ResFiLM) and fed into a Diffusion Policy to generate robust manipulation actions. Extensive experiments on both vision-only and visuo-tactile tasks show that OCRA significantly outperforms existing baselines and ablations, demonstrating its effectiveness for learning from human demonstration videos. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_14401 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | OCRA: Object-Centric Learning with 3D and Tactile Priors for Human-to-Robot Action Transfer Wang, Kuanning Fan, Ke Fu, Yuqian Lin, Siyu Luo, Hu Seita, Daniel Fu, Yanwei Jiang, Yu-Gang Xue, Xiangyang Robotics Computer Vision and Pattern Recognition We present OCRA, an Object-Centric framework for video-based human-to-Robot Action transfer that learns directly from human demonstration videos to enable robust manipulation. Object-centric learning emphasizes task-relevant objects and their interactions while filtering out irrelevant background, providing a natural and scalable way to teach robots. OCRA leverages multi-view RGB videos, the state-of-the-art 3D foundation model VGGT, and advanced detection and segmentation models to reconstruct object-centric 3D point clouds, capturing rich interactions between objects. To handle properties not easily perceived by vision alone, we incorporate tactile priors via a large-scale dataset of over one million tactile images. These 3D and tactile priors are fused through a multimodal module (ResFiLM) and fed into a Diffusion Policy to generate robust manipulation actions. Extensive experiments on both vision-only and visuo-tactile tasks show that OCRA significantly outperforms existing baselines and ablations, demonstrating its effectiveness for learning from human demonstration videos. |
| title | OCRA: Object-Centric Learning with 3D and Tactile Priors for Human-to-Robot Action Transfer |
| topic | Robotics Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2603.14401 |