OmniFit: Multi-modal 3D Body Fitting via Scale-agnostic Dense Landmark Prediction
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910160420601856 |
|---|---|
| author | Cai, Zeyu Xiu, Yuliang Wang, Renke Shao, Zhijing Li, Xiaoben Yu, Siyuan Xu, Chao Liu, Yang Sun, Baigui Yang, Jian Zhang, Zhenyu |
| author_facet | Cai, Zeyu Xiu, Yuliang Wang, Renke Shao, Zhijing Li, Xiaoben Yu, Siyuan Xu, Chao Liu, Yang Sun, Baigui Yang, Jian Zhang, Zhenyu |
| contents | Fitting an underlying body model to 3D clothed human assets has been extensively studied, yet most approaches focus on either single-modal inputs such as point clouds or multi-view images alone, often requiring a known metric scale. This constraint is frequently impractical, especially for AI-generated assets where scale distortion is common. We propose OmniFit, a method that can seamlessly handle diverse multi-modal inputs, including full scans, partial depth observations, and image captures, while remaining scale-agnostic for both real and synthetic assets. Our key innovation is a simple yet effective conditional transformer decoder that directly maps surface points to dense body landmarks, which are then used for SMPL-X parameter fitting. In addition, an optional plug-and-play image adapter incorporates visual cues to compensate for missing geometric information. We further introduce a dedicated scale predictor that rescales subjects to canonical body proportions. OmniFit substantially outperforms state-of-the-art methods by 57.1 to 80.9 percent across daily and loose clothing scenarios. To the best of our knowledge, it is the first body fitting method to surpass multi-view optimization baselines and the first to achieve millimeter-level accuracy on the CAPE and 4D-DRESS benchmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_21575 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | OmniFit: Multi-modal 3D Body Fitting via Scale-agnostic Dense Landmark Prediction Cai, Zeyu Xiu, Yuliang Wang, Renke Shao, Zhijing Li, Xiaoben Yu, Siyuan Xu, Chao Liu, Yang Sun, Baigui Yang, Jian Zhang, Zhenyu Computer Vision and Pattern Recognition Graphics Fitting an underlying body model to 3D clothed human assets has been extensively studied, yet most approaches focus on either single-modal inputs such as point clouds or multi-view images alone, often requiring a known metric scale. This constraint is frequently impractical, especially for AI-generated assets where scale distortion is common. We propose OmniFit, a method that can seamlessly handle diverse multi-modal inputs, including full scans, partial depth observations, and image captures, while remaining scale-agnostic for both real and synthetic assets. Our key innovation is a simple yet effective conditional transformer decoder that directly maps surface points to dense body landmarks, which are then used for SMPL-X parameter fitting. In addition, an optional plug-and-play image adapter incorporates visual cues to compensate for missing geometric information. We further introduce a dedicated scale predictor that rescales subjects to canonical body proportions. OmniFit substantially outperforms state-of-the-art methods by 57.1 to 80.9 percent across daily and loose clothing scenarios. To the best of our knowledge, it is the first body fitting method to surpass multi-view optimization baselines and the first to achieve millimeter-level accuracy on the CAPE and 4D-DRESS benchmarks. |
| title | OmniFit: Multi-modal 3D Body Fitting via Scale-agnostic Dense Landmark Prediction |
| topic | Computer Vision and Pattern Recognition Graphics |
| url | https://arxiv.org/abs/2604.21575 |