SEREP: Semantic Facial Expression Representation for Robust In-the-Wild Capture and Retargeting
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915383653433344 |
|---|---|
| author | Josi, Arthur Hafemann, Luiz Gustavo Dib, Abdallah Got, Emeline Cruz, Rafael M. O. Carbonneau, Marc-Andre |
| author_facet | Josi, Arthur Hafemann, Luiz Gustavo Dib, Abdallah Got, Emeline Cruz, Rafael M. O. Carbonneau, Marc-Andre |
| contents | Monocular facial performance capture in-the-wild is challenging due to varied capture conditions, face shapes, and expressions. Most current methods rely on linear 3D Morphable Models, which represent facial expressions independently of identity at the vertex displacement level. We propose SEREP (Semantic Expression Representation), a model that disentangles expression from identity at the semantic level. We start by learning an expression representation from high-quality 3D data of unpaired facial expressions. Then, we train a model to predict expression from monocular images relying on a novel semi-supervised scheme using low quality synthetic data. In addition, we introduce MultiREX, a benchmark addressing the lack of evaluation resources for the expression capture task. Our experiments show that SEREP outperforms state-of-the-art methods, capturing challenging expressions and transferring them to new identities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_14371 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | SEREP: Semantic Facial Expression Representation for Robust In-the-Wild Capture and Retargeting Josi, Arthur Hafemann, Luiz Gustavo Dib, Abdallah Got, Emeline Cruz, Rafael M. O. Carbonneau, Marc-Andre Computer Vision and Pattern Recognition Graphics Machine Learning Monocular facial performance capture in-the-wild is challenging due to varied capture conditions, face shapes, and expressions. Most current methods rely on linear 3D Morphable Models, which represent facial expressions independently of identity at the vertex displacement level. We propose SEREP (Semantic Expression Representation), a model that disentangles expression from identity at the semantic level. We start by learning an expression representation from high-quality 3D data of unpaired facial expressions. Then, we train a model to predict expression from monocular images relying on a novel semi-supervised scheme using low quality synthetic data. In addition, we introduce MultiREX, a benchmark addressing the lack of evaluation resources for the expression capture task. Our experiments show that SEREP outperforms state-of-the-art methods, capturing challenging expressions and transferring them to new identities. |
| title | SEREP: Semantic Facial Expression Representation for Robust In-the-Wild Capture and Retargeting |
| topic | Computer Vision and Pattern Recognition Graphics Machine Learning |
| url | https://arxiv.org/abs/2412.14371 |