Exploiting Spatial-Temporal Context for Interacting Hand Reconstruction on Monocular RGB Video
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914630442418176 |
|---|---|
| author | Zhao, Weichao Hu, Hezhen Zhou, Wengang li, Li Li, Houqiang |
| author_facet | Zhao, Weichao Hu, Hezhen Zhou, Wengang li, Li Li, Houqiang |
| contents | Reconstructing interacting hands from monocular RGB data is a challenging task, as it involves many interfering factors, e.g. self- and mutual occlusion and similar textures. Previous works only leverage information from a single RGB image without modeling their physically plausible relation, which leads to inferior reconstruction results. In this work, we are dedicated to explicitly exploiting spatial-temporal information to achieve better interacting hand reconstruction. On one hand, we leverage temporal context to complement insufficient information provided by the single frame, and design a novel temporal framework with a temporal constraint for interacting hand motion smoothness. On the other hand, we further propose an interpenetration detection module to produce kinetically plausible interacting hands without physical collisions. Extensive experiments are performed to validate the effectiveness of our proposed framework, which achieves new state-of-the-art performance on public benchmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2308_04074 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Exploiting Spatial-Temporal Context for Interacting Hand Reconstruction on Monocular RGB Video Zhao, Weichao Hu, Hezhen Zhou, Wengang li, Li Li, Houqiang Computer Vision and Pattern Recognition Reconstructing interacting hands from monocular RGB data is a challenging task, as it involves many interfering factors, e.g. self- and mutual occlusion and similar textures. Previous works only leverage information from a single RGB image without modeling their physically plausible relation, which leads to inferior reconstruction results. In this work, we are dedicated to explicitly exploiting spatial-temporal information to achieve better interacting hand reconstruction. On one hand, we leverage temporal context to complement insufficient information provided by the single frame, and design a novel temporal framework with a temporal constraint for interacting hand motion smoothness. On the other hand, we further propose an interpenetration detection module to produce kinetically plausible interacting hands without physical collisions. Extensive experiments are performed to validate the effectiveness of our proposed framework, which achieves new state-of-the-art performance on public benchmarks. |
| title | Exploiting Spatial-Temporal Context for Interacting Hand Reconstruction on Monocular RGB Video |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2308.04074 |