MeshPose: Unifying DensePose and 3D Body Mesh reconstruction
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911918596292608 |
|---|---|
| author | Lê, Eric-Tuan Kakolyris, Antonis Koutras, Petros Tam, Himmy Skordos, Efstratios Papandreou, George Güler, Rıza Alp Kokkinos, Iasonas |
| author_facet | Lê, Eric-Tuan Kakolyris, Antonis Koutras, Petros Tam, Himmy Skordos, Efstratios Papandreou, George Güler, Rıza Alp Kokkinos, Iasonas |
| contents | DensePose provides a pixel-accurate association of images with 3D mesh coordinates, but does not provide a 3D mesh, while Human Mesh Reconstruction (HMR) systems have high 2D reprojection error, as measured by DensePose localization metrics. In this work we introduce MeshPose to jointly tackle DensePose and HMR. For this we first introduce new losses that allow us to use weak DensePose supervision to accurately localize in 2D a subset of the mesh vertices ('VertexPose'). We then lift these vertices to 3D, yielding a low-poly body mesh ('MeshPose'). Our system is trained in an end-to-end manner and is the first HMR method to attain competitive DensePose accuracy, while also being lightweight and amenable to efficient inference, making it suitable for real-time AR applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_10180 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MeshPose: Unifying DensePose and 3D Body Mesh reconstruction Lê, Eric-Tuan Kakolyris, Antonis Koutras, Petros Tam, Himmy Skordos, Efstratios Papandreou, George Güler, Rıza Alp Kokkinos, Iasonas Computer Vision and Pattern Recognition 68 I.2.10 DensePose provides a pixel-accurate association of images with 3D mesh coordinates, but does not provide a 3D mesh, while Human Mesh Reconstruction (HMR) systems have high 2D reprojection error, as measured by DensePose localization metrics. In this work we introduce MeshPose to jointly tackle DensePose and HMR. For this we first introduce new losses that allow us to use weak DensePose supervision to accurately localize in 2D a subset of the mesh vertices ('VertexPose'). We then lift these vertices to 3D, yielding a low-poly body mesh ('MeshPose'). Our system is trained in an end-to-end manner and is the first HMR method to attain competitive DensePose accuracy, while also being lightweight and amenable to efficient inference, making it suitable for real-time AR applications. |
| title | MeshPose: Unifying DensePose and 3D Body Mesh reconstruction |
| topic | Computer Vision and Pattern Recognition 68 I.2.10 |
| url | https://arxiv.org/abs/2406.10180 |