Zero in on Shape: A Generic 2D-3D Instance Similarity Metric learned from Synthetic Data
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2021
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914770619203584 |
|---|---|
| author | Janik, Maciej Gard, Niklas Hilsmann, Anna Eisert, Peter |
| author_facet | Janik, Maciej Gard, Niklas Hilsmann, Anna Eisert, Peter |
| contents | We present a network architecture which compares RGB images and untextured 3D models by the similarity of the represented shape. Our system is optimised for zero-shot retrieval, meaning it can recognise shapes never shown in training. We use a view-based shape descriptor and a siamese network to learn object geometry from pairs of 3D models and 2D images. Due to scarcity of datasets with exact photograph-mesh correspondences, we train our network with only synthetic data. Our experiments investigate the effect of different qualities and quantities of training data on retrieval accuracy and present insights from bridging the domain gap. We show that increasing the variety of synthetic data improves retrieval accuracy and that our system's performance in zero-shot mode can match that of the instance-aware mode, as far as narrowing down the search to the top 10% of objects. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2108_04091 |
| institution | arXiv |
| publishDate | 2021 |
| record_format | arxiv |
| spellingShingle | Zero in on Shape: A Generic 2D-3D Instance Similarity Metric learned from Synthetic Data Janik, Maciej Gard, Niklas Hilsmann, Anna Eisert, Peter Computer Vision and Pattern Recognition We present a network architecture which compares RGB images and untextured 3D models by the similarity of the represented shape. Our system is optimised for zero-shot retrieval, meaning it can recognise shapes never shown in training. We use a view-based shape descriptor and a siamese network to learn object geometry from pairs of 3D models and 2D images. Due to scarcity of datasets with exact photograph-mesh correspondences, we train our network with only synthetic data. Our experiments investigate the effect of different qualities and quantities of training data on retrieval accuracy and present insights from bridging the domain gap. We show that increasing the variety of synthetic data improves retrieval accuracy and that our system's performance in zero-shot mode can match that of the instance-aware mode, as far as narrowing down the search to the top 10% of objects. |
| title | Zero in on Shape: A Generic 2D-3D Instance Similarity Metric learned from Synthetic Data |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2108.04091 |