Scaling Up Dynamic Human-Scene Interaction Modeling
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909209711345664 |
|---|---|
| author | Jiang, Nan Zhang, Zhiyuan Li, Hongjie Ma, Xiaoxuan Wang, Zan Chen, Yixin Liu, Tengyu Zhu, Yixin Huang, Siyuan |
| author_facet | Jiang, Nan Zhang, Zhiyuan Li, Hongjie Ma, Xiaoxuan Wang, Zan Chen, Yixin Liu, Tengyu Zhu, Yixin Huang, Siyuan |
| contents | Confronting the challenges of data scarcity and advanced motion synthesis in human-scene interaction modeling, we introduce the TRUMANS dataset alongside a novel HSI motion synthesis method. TRUMANS stands as the most comprehensive motion-captured HSI dataset currently available, encompassing over 15 hours of human interactions across 100 indoor scenes. It intricately captures whole-body human motions and part-level object dynamics, focusing on the realism of contact. This dataset is further scaled up by transforming physical environments into exact virtual models and applying extensive augmentations to appearance and motion for both humans and objects while maintaining interaction fidelity. Utilizing TRUMANS, we devise a diffusion-based autoregressive model that efficiently generates HSI sequences of any length, taking into account both scene context and intended actions. In experiments, our approach shows remarkable zero-shot generalizability on a range of 3D scene datasets (e.g., PROX, Replica, ScanNet, ScanNet++), producing motions that closely mimic original motion-captured sequences, as confirmed by quantitative experiments and human studies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_08629 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Scaling Up Dynamic Human-Scene Interaction Modeling Jiang, Nan Zhang, Zhiyuan Li, Hongjie Ma, Xiaoxuan Wang, Zan Chen, Yixin Liu, Tengyu Zhu, Yixin Huang, Siyuan Computer Vision and Pattern Recognition Confronting the challenges of data scarcity and advanced motion synthesis in human-scene interaction modeling, we introduce the TRUMANS dataset alongside a novel HSI motion synthesis method. TRUMANS stands as the most comprehensive motion-captured HSI dataset currently available, encompassing over 15 hours of human interactions across 100 indoor scenes. It intricately captures whole-body human motions and part-level object dynamics, focusing on the realism of contact. This dataset is further scaled up by transforming physical environments into exact virtual models and applying extensive augmentations to appearance and motion for both humans and objects while maintaining interaction fidelity. Utilizing TRUMANS, we devise a diffusion-based autoregressive model that efficiently generates HSI sequences of any length, taking into account both scene context and intended actions. In experiments, our approach shows remarkable zero-shot generalizability on a range of 3D scene datasets (e.g., PROX, Replica, ScanNet, ScanNet++), producing motions that closely mimic original motion-captured sequences, as confirmed by quantitative experiments and human studies. |
| title | Scaling Up Dynamic Human-Scene Interaction Modeling |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2403.08629 |