KP-RED: Exploiting Semantic Keypoints for Joint 3D Shape Retrieval and Deformation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910725567414272 |
|---|---|
| author | Zhang, Ruida Zhang, Chenyangguang Di, Yan Manhardt, Fabian Liu, Xingyu Tombari, Federico Ji, Xiangyang |
| author_facet | Zhang, Ruida Zhang, Chenyangguang Di, Yan Manhardt, Fabian Liu, Xingyu Tombari, Federico Ji, Xiangyang |
| contents | In this paper, we present KP-RED, a unified KeyPoint-driven REtrieval and Deformation framework that takes object scans as input and jointly retrieves and deforms the most geometrically similar CAD models from a pre-processed database to tightly match the target. Unlike existing dense matching based methods that typically struggle with noisy partial scans, we propose to leverage category-consistent sparse keypoints to naturally handle both full and partial object scans. Specifically, we first employ a lightweight retrieval module to establish a keypoint-based embedding space, measuring the similarity among objects by dynamically aggregating deformation-aware local-global features around extracted keypoints. Objects that are close in the embedding space are considered similar in geometry. Then we introduce the neural cage-based deformation module that estimates the influence vector of each keypoint upon cage vertices inside its local support region to control the deformation of the retrieved shape. Extensive experiments on the synthetic dataset PartNet and the real-world dataset Scan2CAD demonstrate that KP-RED surpasses existing state-of-the-art approaches by a large margin. Codes and trained models are released on https://github.com/lolrudy/KP-RED. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_10099 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | KP-RED: Exploiting Semantic Keypoints for Joint 3D Shape Retrieval and Deformation Zhang, Ruida Zhang, Chenyangguang Di, Yan Manhardt, Fabian Liu, Xingyu Tombari, Federico Ji, Xiangyang Computer Vision and Pattern Recognition In this paper, we present KP-RED, a unified KeyPoint-driven REtrieval and Deformation framework that takes object scans as input and jointly retrieves and deforms the most geometrically similar CAD models from a pre-processed database to tightly match the target. Unlike existing dense matching based methods that typically struggle with noisy partial scans, we propose to leverage category-consistent sparse keypoints to naturally handle both full and partial object scans. Specifically, we first employ a lightweight retrieval module to establish a keypoint-based embedding space, measuring the similarity among objects by dynamically aggregating deformation-aware local-global features around extracted keypoints. Objects that are close in the embedding space are considered similar in geometry. Then we introduce the neural cage-based deformation module that estimates the influence vector of each keypoint upon cage vertices inside its local support region to control the deformation of the retrieved shape. Extensive experiments on the synthetic dataset PartNet and the real-world dataset Scan2CAD demonstrate that KP-RED surpasses existing state-of-the-art approaches by a large margin. Codes and trained models are released on https://github.com/lolrudy/KP-RED. |
| title | KP-RED: Exploiting Semantic Keypoints for Joint 3D Shape Retrieval and Deformation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2403.10099 |