SemanticFeels: Semantic Labeling during In-Hand Manipulation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912906228006912 |
|---|---|
| author | Khalil, Anas Al Shikh Qi, Haozhi Calandra, Roberto |
| author_facet | Khalil, Anas Al Shikh Qi, Haozhi Calandra, Roberto |
| contents | As robots become increasingly integrated into everyday tasks, their ability to perceive both the shape and properties of objects during in-hand manipulation becomes critical for adaptive and intelligent behavior. We present SemanticFeels, an extension of the NeuralFeels framework that integrates semantic labeling with neural implicit shape representation, from vision and touch. To illustrate its application, we focus on material classification: high-resolution Digit tactile readings are processed by a fine-tuned EfficientNet-B0 convolutional neural network (CNN) to generate local material predictions, which are then embedded into an augmented signed distance field (SDF) network that jointly predicts geometry and continuous material regions. Experimental results show that the system achieves a high correspondence between predicted and actual materials on both single- and multi-material objects, with an average matching accuracy of 79.87% across multiple manipulation trials on a multi-material object. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_14099 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SemanticFeels: Semantic Labeling during In-Hand Manipulation Khalil, Anas Al Shikh Qi, Haozhi Calandra, Roberto Robotics Artificial Intelligence Computer Vision and Pattern Recognition 68T45, 68T05 I.2.9; I.2.10; I.4.8; I.2.6 As robots become increasingly integrated into everyday tasks, their ability to perceive both the shape and properties of objects during in-hand manipulation becomes critical for adaptive and intelligent behavior. We present SemanticFeels, an extension of the NeuralFeels framework that integrates semantic labeling with neural implicit shape representation, from vision and touch. To illustrate its application, we focus on material classification: high-resolution Digit tactile readings are processed by a fine-tuned EfficientNet-B0 convolutional neural network (CNN) to generate local material predictions, which are then embedded into an augmented signed distance field (SDF) network that jointly predicts geometry and continuous material regions. Experimental results show that the system achieves a high correspondence between predicted and actual materials on both single- and multi-material objects, with an average matching accuracy of 79.87% across multiple manipulation trials on a multi-material object. |
| title | SemanticFeels: Semantic Labeling during In-Hand Manipulation |
| topic | Robotics Artificial Intelligence Computer Vision and Pattern Recognition 68T45, 68T05 I.2.9; I.2.10; I.4.8; I.2.6 |
| url | https://arxiv.org/abs/2602.14099 |