MultiPhys: Multi-Person Physics-aware 3D Motion Estimation
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914759810482176 |
|---|---|
| author | Ugrinovic, Nicolas Pan, Boxiao Pavlakos, Georgios Paschalidou, Despoina Shen, Bokui Sanchez-Riera, Jordi Moreno-Noguer, Francesc Guibas, Leonidas |
| author_facet | Ugrinovic, Nicolas Pan, Boxiao Pavlakos, Georgios Paschalidou, Despoina Shen, Bokui Sanchez-Riera, Jordi Moreno-Noguer, Francesc Guibas, Leonidas |
| contents | We introduce MultiPhys, a method designed for recovering multi-person motion from monocular videos. Our focus lies in capturing coherent spatial placement between pairs of individuals across varying degrees of engagement. MultiPhys, being physically aware, exhibits robustness to jittering and occlusions, and effectively eliminates penetration issues between the two individuals. We devise a pipeline in which the motion estimated by a kinematic-based method is fed into a physics simulator in an autoregressive manner. We introduce distinct components that enable our model to harness the simulator's properties without compromising the accuracy of the kinematic estimates. This results in final motion estimates that are both kinematically coherent and physically compliant. Extensive evaluations on three challenging datasets characterized by substantial inter-person interaction show that our method significantly reduces errors associated with penetration and foot skating, while performing competitively with the state-of-the-art on motion accuracy and smoothness. Results and code can be found on our project page (http://www.iri.upc.edu/people/nugrinovic/multiphys/). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_11987 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MultiPhys: Multi-Person Physics-aware 3D Motion Estimation Ugrinovic, Nicolas Pan, Boxiao Pavlakos, Georgios Paschalidou, Despoina Shen, Bokui Sanchez-Riera, Jordi Moreno-Noguer, Francesc Guibas, Leonidas Computer Vision and Pattern Recognition We introduce MultiPhys, a method designed for recovering multi-person motion from monocular videos. Our focus lies in capturing coherent spatial placement between pairs of individuals across varying degrees of engagement. MultiPhys, being physically aware, exhibits robustness to jittering and occlusions, and effectively eliminates penetration issues between the two individuals. We devise a pipeline in which the motion estimated by a kinematic-based method is fed into a physics simulator in an autoregressive manner. We introduce distinct components that enable our model to harness the simulator's properties without compromising the accuracy of the kinematic estimates. This results in final motion estimates that are both kinematically coherent and physically compliant. Extensive evaluations on three challenging datasets characterized by substantial inter-person interaction show that our method significantly reduces errors associated with penetration and foot skating, while performing competitively with the state-of-the-art on motion accuracy and smoothness. Results and code can be found on our project page (http://www.iri.upc.edu/people/nugrinovic/multiphys/). |
| title | MultiPhys: Multi-Person Physics-aware 3D Motion Estimation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2404.11987 |