Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2312.01068 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917632410648576 |
|---|---|
| author | Tang, Jiapeng Dai, Angela Nie, Yinyu Markhasin, Lev Thies, Justus Niessner, Matthias |
| author_facet | Tang, Jiapeng Dai, Angela Nie, Yinyu Markhasin, Lev Thies, Justus Niessner, Matthias |
| contents | We introduce Diffusion Parametric Head Models (DPHMs), a generative model that enables robust volumetric head reconstruction and tracking from monocular depth sequences. While recent volumetric head models, such as NPHMs, can now excel in representing high-fidelity head geometries, tracking and reconstructing heads from real-world single-view depth sequences remains very challenging, as the fitting to partial and noisy observations is underconstrained. To tackle these challenges, we propose a latent diffusion-based prior to regularize volumetric head reconstruction and tracking. This prior-based regularizer effectively constrains the identity and expression codes to lie on the underlying latent manifold which represents plausible head shapes. To evaluate the effectiveness of the diffusion-based prior, we collect a dataset of monocular Kinect sequences consisting of various complex facial expression motions and rapid transitions. We compare our method to state-of-the-art tracking methods and demonstrate improved head identity reconstruction as well as robust expression tracking. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_01068 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | DPHMs: Diffusion Parametric Head Models for Depth-based Tracking Tang, Jiapeng Dai, Angela Nie, Yinyu Markhasin, Lev Thies, Justus Niessner, Matthias Computer Vision and Pattern Recognition We introduce Diffusion Parametric Head Models (DPHMs), a generative model that enables robust volumetric head reconstruction and tracking from monocular depth sequences. While recent volumetric head models, such as NPHMs, can now excel in representing high-fidelity head geometries, tracking and reconstructing heads from real-world single-view depth sequences remains very challenging, as the fitting to partial and noisy observations is underconstrained. To tackle these challenges, we propose a latent diffusion-based prior to regularize volumetric head reconstruction and tracking. This prior-based regularizer effectively constrains the identity and expression codes to lie on the underlying latent manifold which represents plausible head shapes. To evaluate the effectiveness of the diffusion-based prior, we collect a dataset of monocular Kinect sequences consisting of various complex facial expression motions and rapid transitions. We compare our method to state-of-the-art tracking methods and demonstrate improved head identity reconstruction as well as robust expression tracking. |
| title | DPHMs: Diffusion Parametric Head Models for Depth-based Tracking |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2312.01068 |