Look Ma, no markers: holistic performance capture without the hassle
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916439905009664 |
|---|---|
| author | Hewitt, Charlie Saleh, Fatemeh Aliakbarian, Sadegh Petikam, Lohit Rezaeifar, Shideh Florentin, Louis Hosenie, Zafiirah Cashman, Thomas J Valentin, Julien Cosker, Darren Baltrusaitis, Tadas |
| author_facet | Hewitt, Charlie Saleh, Fatemeh Aliakbarian, Sadegh Petikam, Lohit Rezaeifar, Shideh Florentin, Louis Hosenie, Zafiirah Cashman, Thomas J Valentin, Julien Cosker, Darren Baltrusaitis, Tadas |
| contents | We tackle the problem of highly-accurate, holistic performance capture for the face, body and hands simultaneously. Motion-capture technologies used in film and game production typically focus only on face, body or hand capture independently, involve complex and expensive hardware and a high degree of manual intervention from skilled operators. While machine-learning-based approaches exist to overcome these problems, they usually only support a single camera, often operate on a single part of the body, do not produce precise world-space results, and rarely generalize outside specific contexts. In this work, we introduce the first technique for marker-free, high-quality reconstruction of the complete human body, including eyes and tongue, without requiring any calibration, manual intervention or custom hardware. Our approach produces stable world-space results from arbitrary camera rigs as well as supporting varied capture environments and clothing. We achieve this through a hybrid approach that leverages machine learning models trained exclusively on synthetic data and powerful parametric models of human shape and motion. We evaluate our method on a number of body, face and hand reconstruction benchmarks and demonstrate state-of-the-art results that generalize on diverse datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_11520 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Look Ma, no markers: holistic performance capture without the hassle Hewitt, Charlie Saleh, Fatemeh Aliakbarian, Sadegh Petikam, Lohit Rezaeifar, Shideh Florentin, Louis Hosenie, Zafiirah Cashman, Thomas J Valentin, Julien Cosker, Darren Baltrusaitis, Tadas Computer Vision and Pattern Recognition Graphics We tackle the problem of highly-accurate, holistic performance capture for the face, body and hands simultaneously. Motion-capture technologies used in film and game production typically focus only on face, body or hand capture independently, involve complex and expensive hardware and a high degree of manual intervention from skilled operators. While machine-learning-based approaches exist to overcome these problems, they usually only support a single camera, often operate on a single part of the body, do not produce precise world-space results, and rarely generalize outside specific contexts. In this work, we introduce the first technique for marker-free, high-quality reconstruction of the complete human body, including eyes and tongue, without requiring any calibration, manual intervention or custom hardware. Our approach produces stable world-space results from arbitrary camera rigs as well as supporting varied capture environments and clothing. We achieve this through a hybrid approach that leverages machine learning models trained exclusively on synthetic data and powerful parametric models of human shape and motion. We evaluate our method on a number of body, face and hand reconstruction benchmarks and demonstrate state-of-the-art results that generalize on diverse datasets. |
| title | Look Ma, no markers: holistic performance capture without the hassle |
| topic | Computer Vision and Pattern Recognition Graphics |
| url | https://arxiv.org/abs/2410.11520 |