HandGCAT: Occlusion-Robust 3D Hand Mesh Reconstruction from Monocular Images
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917611869044736 |
|---|---|
| author | Wang, Shuaibing Wang, Shunli Yang, Dingkang Li, Mingcheng Qian, Ziyun Su, Liuzhen Zhang, Lihua |
| author_facet | Wang, Shuaibing Wang, Shunli Yang, Dingkang Li, Mingcheng Qian, Ziyun Su, Liuzhen Zhang, Lihua |
| contents | We propose a robust and accurate method for reconstructing 3D hand mesh from monocular images. This is a very challenging problem, as hands are often severely occluded by objects. Previous works often have disregarded 2D hand pose information, which contains hand prior knowledge that is strongly correlated with occluded regions. Thus, in this work, we propose a novel 3D hand mesh reconstruction network HandGCAT, that can fully exploit hand prior as compensation information to enhance occluded region features. Specifically, we designed the Knowledge-Guided Graph Convolution (KGC) module and the Cross-Attention Transformer (CAT) module. KGC extracts hand prior information from 2D hand pose by graph convolution. CAT fuses hand prior into occluded regions by considering their high correlation. Extensive experiments on popular datasets with challenging hand-object occlusions, such as HO3D v2, HO3D v3, and DexYCB demonstrate that our HandGCAT reaches state-of-the-art performance. The code is available at https://github.com/heartStrive/HandGCAT. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_07912 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | HandGCAT: Occlusion-Robust 3D Hand Mesh Reconstruction from Monocular Images Wang, Shuaibing Wang, Shunli Yang, Dingkang Li, Mingcheng Qian, Ziyun Su, Liuzhen Zhang, Lihua Computer Vision and Pattern Recognition We propose a robust and accurate method for reconstructing 3D hand mesh from monocular images. This is a very challenging problem, as hands are often severely occluded by objects. Previous works often have disregarded 2D hand pose information, which contains hand prior knowledge that is strongly correlated with occluded regions. Thus, in this work, we propose a novel 3D hand mesh reconstruction network HandGCAT, that can fully exploit hand prior as compensation information to enhance occluded region features. Specifically, we designed the Knowledge-Guided Graph Convolution (KGC) module and the Cross-Attention Transformer (CAT) module. KGC extracts hand prior information from 2D hand pose by graph convolution. CAT fuses hand prior into occluded regions by considering their high correlation. Extensive experiments on popular datasets with challenging hand-object occlusions, such as HO3D v2, HO3D v3, and DexYCB demonstrate that our HandGCAT reaches state-of-the-art performance. The code is available at https://github.com/heartStrive/HandGCAT. |
| title | HandGCAT: Occlusion-Robust 3D Hand Mesh Reconstruction from Monocular Images |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2403.07912 |