Diffusion-HPC: Synthetic Data Generation for Human Mesh Recovery in Challenging Domains
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910283931320320 |
|---|---|
| author | Weng, Zhenzhen Bravo-Sánchez, Laura Yeung-Levy, Serena |
| author_facet | Weng, Zhenzhen Bravo-Sánchez, Laura Yeung-Levy, Serena |
| contents | Recent text-to-image generative models have exhibited remarkable abilities in generating high-fidelity and photo-realistic images. However, despite the visually impressive results, these models often struggle to preserve plausible human structure in the generations. Due to this reason, while generative models have shown promising results in aiding downstream image recognition tasks by generating large volumes of synthetic data, they are not suitable for improving downstream human pose perception and understanding. In this work, we propose a Diffusion model with Human Pose Correction (Diffusion-HPC), a text-conditioned method that generates photo-realistic images with plausible posed humans by injecting prior knowledge about human body structure. Our generated images are accompanied by 3D meshes that serve as ground truths for improving Human Mesh Recovery tasks, where a shortage of 3D training data has long been an issue. Furthermore, we show that Diffusion-HPC effectively improves the realism of human generations under varying conditioning strategies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2303_09541 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Diffusion-HPC: Synthetic Data Generation for Human Mesh Recovery in Challenging Domains Weng, Zhenzhen Bravo-Sánchez, Laura Yeung-Levy, Serena Computer Vision and Pattern Recognition Recent text-to-image generative models have exhibited remarkable abilities in generating high-fidelity and photo-realistic images. However, despite the visually impressive results, these models often struggle to preserve plausible human structure in the generations. Due to this reason, while generative models have shown promising results in aiding downstream image recognition tasks by generating large volumes of synthetic data, they are not suitable for improving downstream human pose perception and understanding. In this work, we propose a Diffusion model with Human Pose Correction (Diffusion-HPC), a text-conditioned method that generates photo-realistic images with plausible posed humans by injecting prior knowledge about human body structure. Our generated images are accompanied by 3D meshes that serve as ground truths for improving Human Mesh Recovery tasks, where a shortage of 3D training data has long been an issue. Furthermore, we show that Diffusion-HPC effectively improves the realism of human generations under varying conditioning strategies. |
| title | Diffusion-HPC: Synthetic Data Generation for Human Mesh Recovery in Challenging Domains |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2303.09541 |