Affordance segmentation of hand-occluded containers from exocentric images
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2023
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910933589164032 |
|---|---|
| author | Apicella, Tommaso Xompero, Alessio Ragusa, Edoardo Berta, Riccardo Cavallaro, Andrea Gastaldo, Paolo |
| author_facet | Apicella, Tommaso Xompero, Alessio Ragusa, Edoardo Berta, Riccardo Cavallaro, Andrea Gastaldo, Paolo |
| contents | Visual affordance segmentation identifies the surfaces of an object an agent can interact with. Common challenges for the identification of affordances are the variety of the geometry and physical properties of these surfaces as well as occlusions. In this paper, we focus on occlusions of an object that is hand-held by a person manipulating it. To address this challenge, we propose an affordance segmentation model that uses auxiliary branches to process the object and hand regions separately. The proposed model learns affordance features under hand-occlusion by weighting the feature map through hand and object segmentation. To train the model, we annotated the visual affordances of an existing dataset with mixed-reality images of hand-held containers in third-person (exocentric) images. Experiments on both real and mixed-reality images show that our model achieves better affordance segmentation and generalisation than existing models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2308_11233 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Affordance segmentation of hand-occluded containers from exocentric images Apicella, Tommaso Xompero, Alessio Ragusa, Edoardo Berta, Riccardo Cavallaro, Andrea Gastaldo, Paolo Computer Vision and Pattern Recognition Visual affordance segmentation identifies the surfaces of an object an agent can interact with. Common challenges for the identification of affordances are the variety of the geometry and physical properties of these surfaces as well as occlusions. In this paper, we focus on occlusions of an object that is hand-held by a person manipulating it. To address this challenge, we propose an affordance segmentation model that uses auxiliary branches to process the object and hand regions separately. The proposed model learns affordance features under hand-occlusion by weighting the feature map through hand and object segmentation. To train the model, we annotated the visual affordances of an existing dataset with mixed-reality images of hand-held containers in third-person (exocentric) images. Experiments on both real and mixed-reality images show that our model achieves better affordance segmentation and generalisation than existing models. |
| title | Affordance segmentation of hand-occluded containers from exocentric images |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2308.11233 |