Robotic Perception with a Large Tactile-Vision-Language Model for Physical Property Inference
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909658740948992 |
|---|---|
| author | Guo, Zexiang Chen, Hengxiang Mai, Xinheng Qiu, Qiusang Ma, Gan Kappassov, Zhanat Li, Qiang Chen, Nutan |
| author_facet | Guo, Zexiang Chen, Hengxiang Mai, Xinheng Qiu, Qiusang Ma, Gan Kappassov, Zhanat Li, Qiang Chen, Nutan |
| contents | Inferring physical properties can significantly enhance robotic manipulation by enabling robots to handle objects safely and efficiently through adaptive grasping strategies. Previous approaches have typically relied on either tactile or visual data, limiting their ability to fully capture properties. We introduce a novel cross-modal perception framework that integrates visual observations with tactile representations within a multimodal vision-language model. Our physical reasoning framework, which employs a hierarchical feature alignment mechanism and a refined prompting strategy, enables our model to make property-specific predictions that strongly correlate with ground-truth measurements. Evaluated on 35 diverse objects, our approach outperforms existing baselines and demonstrates strong zero-shot generalization. Keywords: tactile perception, visual-tactile fusion, physical property inference, multimodal integration, robot perception |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_19303 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Robotic Perception with a Large Tactile-Vision-Language Model for Physical Property Inference Guo, Zexiang Chen, Hengxiang Mai, Xinheng Qiu, Qiusang Ma, Gan Kappassov, Zhanat Li, Qiang Chen, Nutan Robotics Inferring physical properties can significantly enhance robotic manipulation by enabling robots to handle objects safely and efficiently through adaptive grasping strategies. Previous approaches have typically relied on either tactile or visual data, limiting their ability to fully capture properties. We introduce a novel cross-modal perception framework that integrates visual observations with tactile representations within a multimodal vision-language model. Our physical reasoning framework, which employs a hierarchical feature alignment mechanism and a refined prompting strategy, enables our model to make property-specific predictions that strongly correlate with ground-truth measurements. Evaluated on 35 diverse objects, our approach outperforms existing baselines and demonstrates strong zero-shot generalization. Keywords: tactile perception, visual-tactile fusion, physical property inference, multimodal integration, robot perception |
| title | Robotic Perception with a Large Tactile-Vision-Language Model for Physical Property Inference |
| topic | Robotics |
| url | https://arxiv.org/abs/2506.19303 |