DiffMesh: A Motion-aware Diffusion Framework for Human Mesh Recovery from Videos
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911110383271936 |
|---|---|
| author | Zheng, Ce Liu, Xianpeng Peng, Qucheng Wu, Tianfu Wang, Pu Chen, Chen |
| author_facet | Zheng, Ce Liu, Xianpeng Peng, Qucheng Wu, Tianfu Wang, Pu Chen, Chen |
| contents | Human mesh recovery (HMR) provides rich human body information for various real-world applications. While image-based HMR methods have achieved impressive results, they often struggle to recover humans in dynamic scenarios, leading to temporal inconsistencies and non-smooth 3D motion predictions due to the absence of human motion. In contrast, video-based approaches leverage temporal information to mitigate this issue. In this paper, we present DiffMesh, an innovative motion-aware Diffusion-like framework for video-based HMR. DiffMesh establishes a bridge between diffusion models and human motion, efficiently generating accurate and smooth output mesh sequences by incorporating human motion within the forward process and reverse process in the diffusion model. Extensive experiments are conducted on the widely used datasets (Human3.6M \cite{h36m_pami} and 3DPW \cite{pw3d2018}), which demonstrate the effectiveness and efficiency of our DiffMesh. Visual comparisons in real-world scenarios further highlight DiffMesh's suitability for practical applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2303_13397 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | DiffMesh: A Motion-aware Diffusion Framework for Human Mesh Recovery from Videos Zheng, Ce Liu, Xianpeng Peng, Qucheng Wu, Tianfu Wang, Pu Chen, Chen Computer Vision and Pattern Recognition Artificial Intelligence Human-Computer Interaction Multimedia Human mesh recovery (HMR) provides rich human body information for various real-world applications. While image-based HMR methods have achieved impressive results, they often struggle to recover humans in dynamic scenarios, leading to temporal inconsistencies and non-smooth 3D motion predictions due to the absence of human motion. In contrast, video-based approaches leverage temporal information to mitigate this issue. In this paper, we present DiffMesh, an innovative motion-aware Diffusion-like framework for video-based HMR. DiffMesh establishes a bridge between diffusion models and human motion, efficiently generating accurate and smooth output mesh sequences by incorporating human motion within the forward process and reverse process in the diffusion model. Extensive experiments are conducted on the widely used datasets (Human3.6M \cite{h36m_pami} and 3DPW \cite{pw3d2018}), which demonstrate the effectiveness and efficiency of our DiffMesh. Visual comparisons in real-world scenarios further highlight DiffMesh's suitability for practical applications. |
| title | DiffMesh: A Motion-aware Diffusion Framework for Human Mesh Recovery from Videos |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Human-Computer Interaction Multimedia |
| url | https://arxiv.org/abs/2303.13397 |