StackFLOW: Monocular Human-Object Reconstruction by Stacked Normalizing Flow with Offset
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910546125651968 |
|---|---|
| author | Huo, Chaofan Shi, Ye Ma, Yuexin Xu, Lan Yu, Jingyi Wang, Jingya |
| author_facet | Huo, Chaofan Shi, Ye Ma, Yuexin Xu, Lan Yu, Jingyi Wang, Jingya |
| contents | Modeling and capturing the 3D spatial arrangement of the human and the object is the key to perceiving 3D human-object interaction from monocular images. In this work, we propose to use the Human-Object Offset between anchors which are densely sampled from the surface of human mesh and object mesh to represent human-object spatial relation. Compared with previous works which use contact map or implicit distance filed to encode 3D human-object spatial relations, our method is a simple and efficient way to encode the highly detailed spatial correlation between the human and object. Based on this representation, we propose Stacked Normalizing Flow (StackFLOW) to infer the posterior distribution of human-object spatial relations from the image. During the optimization stage, we finetune the human body pose and object 6D pose by maximizing the likelihood of samples based on this posterior distribution and minimizing the 2D-3D corresponding reprojection loss. Extensive experimental results show that our method achieves impressive results on two challenging benchmarks, BEHAVE and InterCap datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_20545 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | StackFLOW: Monocular Human-Object Reconstruction by Stacked Normalizing Flow with Offset Huo, Chaofan Shi, Ye Ma, Yuexin Xu, Lan Yu, Jingyi Wang, Jingya Computer Vision and Pattern Recognition Graphics Modeling and capturing the 3D spatial arrangement of the human and the object is the key to perceiving 3D human-object interaction from monocular images. In this work, we propose to use the Human-Object Offset between anchors which are densely sampled from the surface of human mesh and object mesh to represent human-object spatial relation. Compared with previous works which use contact map or implicit distance filed to encode 3D human-object spatial relations, our method is a simple and efficient way to encode the highly detailed spatial correlation between the human and object. Based on this representation, we propose Stacked Normalizing Flow (StackFLOW) to infer the posterior distribution of human-object spatial relations from the image. During the optimization stage, we finetune the human body pose and object 6D pose by maximizing the likelihood of samples based on this posterior distribution and minimizing the 2D-3D corresponding reprojection loss. Extensive experimental results show that our method achieves impressive results on two challenging benchmarks, BEHAVE and InterCap datasets. |
| title | StackFLOW: Monocular Human-Object Reconstruction by Stacked Normalizing Flow with Offset |
| topic | Computer Vision and Pattern Recognition Graphics |
| url | https://arxiv.org/abs/2407.20545 |