SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913449536126976 |
|---|---|
| author | Cai, Zhongang Yin, Wanqi Zeng, Ailing Wei, Chen Sun, Qingping Wang, Yanjun Pang, Hui En Mei, Haiyi Zhang, Mingyuan Zhang, Lei Loy, Chen Change Yang, Lei Liu, Ziwei |
| author_facet | Cai, Zhongang Yin, Wanqi Zeng, Ailing Wei, Chen Sun, Qingping Wang, Yanjun Pang, Hui En Mei, Haiyi Zhang, Mingyuan Zhang, Lei Loy, Chen Change Yang, Lei Liu, Ziwei |
| contents | Expressive human pose and shape estimation (EHPS) unifies body, hands, and face motion capture with numerous applications. Despite encouraging progress, current state-of-the-art methods still depend largely on a confined set of training datasets. In this work, we investigate scaling up EHPS towards the first generalist foundation model (dubbed SMPLer-X), with up to ViT-Huge as the backbone and training with up to 4.5M instances from diverse data sources. With big data and the large model, SMPLer-X exhibits strong performance across diverse test benchmarks and excellent transferability to even unseen environments. 1) For the data scaling, we perform a systematic investigation on 32 EHPS datasets, including a wide range of scenarios that a model trained on any single dataset cannot handle. More importantly, capitalizing on insights obtained from the extensive benchmarking process, we optimize our training scheme and select datasets that lead to a significant leap in EHPS capabilities. 2) For the model scaling, we take advantage of vision transformers to study the scaling law of model sizes in EHPS. Moreover, our finetuning strategy turn SMPLer-X into specialist models, allowing them to achieve further performance boosts. Notably, our foundation model SMPLer-X consistently delivers state-of-the-art results on seven benchmarks such as AGORA (107.2 mm NMVE), UBody (57.4 mm PVE), EgoBody (63.6 mm PVE), and EHF (62.3 mm PVE without finetuning). Homepage: https://caizhongang.github.io/projects/SMPLer-X/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_17448 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimation Cai, Zhongang Yin, Wanqi Zeng, Ailing Wei, Chen Sun, Qingping Wang, Yanjun Pang, Hui En Mei, Haiyi Zhang, Mingyuan Zhang, Lei Loy, Chen Change Yang, Lei Liu, Ziwei Computer Vision and Pattern Recognition Expressive human pose and shape estimation (EHPS) unifies body, hands, and face motion capture with numerous applications. Despite encouraging progress, current state-of-the-art methods still depend largely on a confined set of training datasets. In this work, we investigate scaling up EHPS towards the first generalist foundation model (dubbed SMPLer-X), with up to ViT-Huge as the backbone and training with up to 4.5M instances from diverse data sources. With big data and the large model, SMPLer-X exhibits strong performance across diverse test benchmarks and excellent transferability to even unseen environments. 1) For the data scaling, we perform a systematic investigation on 32 EHPS datasets, including a wide range of scenarios that a model trained on any single dataset cannot handle. More importantly, capitalizing on insights obtained from the extensive benchmarking process, we optimize our training scheme and select datasets that lead to a significant leap in EHPS capabilities. 2) For the model scaling, we take advantage of vision transformers to study the scaling law of model sizes in EHPS. Moreover, our finetuning strategy turn SMPLer-X into specialist models, allowing them to achieve further performance boosts. Notably, our foundation model SMPLer-X consistently delivers state-of-the-art results on seven benchmarks such as AGORA (107.2 mm NMVE), UBody (57.4 mm PVE), EgoBody (63.6 mm PVE), and EHF (62.3 mm PVE without finetuning). Homepage: https://caizhongang.github.io/projects/SMPLer-X/ |
| title | SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2309.17448 |