From 2D Images to 3D Model:Weakly Supervised Multi-View Face Reconstruction with Deep Fusion
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2022
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916540529508352 |
|---|---|
| author | Zhao, Weiguang Yang, Chaolong Ye, Jianan Zhang, Rui Yan, Yuyao Yang, Xi Dong, Bin Hussain, Amir Huang, Kaizhu |
| author_facet | Zhao, Weiguang Yang, Chaolong Ye, Jianan Zhang, Rui Yan, Yuyao Yang, Xi Dong, Bin Hussain, Amir Huang, Kaizhu |
| contents | While weakly supervised multi-view face reconstruction (MVR) is garnering increased attention, one critical issue still remains open: how to effectively interact and fuse multiple image information to reconstruct high-precision 3D models. In this regard, we propose a novel pipeline called Deep Fusion MVR (DF-MVR) to explore the feature correspondences between multi-view images and reconstruct high-precision 3D faces. Specifically, we present a novel multi-view feature fusion backbone that utilizes face masks to align features from multiple encoders and integrates one multi-layer attention mechanism to enhance feature interaction and fusion, resulting in one unified facial representation. Additionally, we develop one concise face mask mechanism that facilitates multi-view feature fusion and facial reconstruction by identifying common areas and guiding the network's focus on critical facial features (e.g., eyes, brows, nose, and mouth). Experiments on Pixel-Face and Bosphorus datasets indicate the superiority of our model. Without 3D annotation, DF-MVR achieves 5.2% and 3.0% RMSE improvement over the existing weakly supervised MVRs respectively on Pixel-Face and Bosphorus dataset. Code will be available publicly at https://github.com/weiguangzhao/DF_MVR. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2204_03842 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | From 2D Images to 3D Model:Weakly Supervised Multi-View Face Reconstruction with Deep Fusion Zhao, Weiguang Yang, Chaolong Ye, Jianan Zhang, Rui Yan, Yuyao Yang, Xi Dong, Bin Hussain, Amir Huang, Kaizhu Computer Vision and Pattern Recognition While weakly supervised multi-view face reconstruction (MVR) is garnering increased attention, one critical issue still remains open: how to effectively interact and fuse multiple image information to reconstruct high-precision 3D models. In this regard, we propose a novel pipeline called Deep Fusion MVR (DF-MVR) to explore the feature correspondences between multi-view images and reconstruct high-precision 3D faces. Specifically, we present a novel multi-view feature fusion backbone that utilizes face masks to align features from multiple encoders and integrates one multi-layer attention mechanism to enhance feature interaction and fusion, resulting in one unified facial representation. Additionally, we develop one concise face mask mechanism that facilitates multi-view feature fusion and facial reconstruction by identifying common areas and guiding the network's focus on critical facial features (e.g., eyes, brows, nose, and mouth). Experiments on Pixel-Face and Bosphorus datasets indicate the superiority of our model. Without 3D annotation, DF-MVR achieves 5.2% and 3.0% RMSE improvement over the existing weakly supervised MVRs respectively on Pixel-Face and Bosphorus dataset. Code will be available publicly at https://github.com/weiguangzhao/DF_MVR. |
| title | From 2D Images to 3D Model:Weakly Supervised Multi-View Face Reconstruction with Deep Fusion |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2204.03842 |