Estimating Body and Hand Motion in an Ego-sensed World
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909431214637056 |
|---|---|
| author | Yi, Brent Ye, Vickie Zheng, Maya Li, Yunqi Müller, Lea Pavlakos, Georgios Ma, Yi Malik, Jitendra Kanazawa, Angjoo |
| author_facet | Yi, Brent Ye, Vickie Zheng, Maya Li, Yunqi Müller, Lea Pavlakos, Georgios Ma, Yi Malik, Jitendra Kanazawa, Angjoo |
| contents | We present EgoAllo, a system for human motion estimation from a head-mounted device. Using only egocentric SLAM poses and images, EgoAllo guides sampling from a conditional diffusion model to estimate 3D body pose, height, and hand parameters that capture a device wearer's actions in the allocentric coordinate frame of the scene. To achieve this, our key insight is in representation: we propose spatial and temporal invariance criteria for improving model performance, from which we derive a head motion conditioning parameterization that improves estimation by up to 18%. We also show how the bodies estimated by our system can improve hand estimation: the resulting kinematic and temporal constraints can reduce world-frame errors in single-frame estimates by 40%. Project page: https://egoallo.github.io/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_03665 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Estimating Body and Hand Motion in an Ego-sensed World Yi, Brent Ye, Vickie Zheng, Maya Li, Yunqi Müller, Lea Pavlakos, Georgios Ma, Yi Malik, Jitendra Kanazawa, Angjoo Computer Vision and Pattern Recognition Artificial Intelligence We present EgoAllo, a system for human motion estimation from a head-mounted device. Using only egocentric SLAM poses and images, EgoAllo guides sampling from a conditional diffusion model to estimate 3D body pose, height, and hand parameters that capture a device wearer's actions in the allocentric coordinate frame of the scene. To achieve this, our key insight is in representation: we propose spatial and temporal invariance criteria for improving model performance, from which we derive a head motion conditioning parameterization that improves estimation by up to 18%. We also show how the bodies estimated by our system can improve hand estimation: the resulting kinematic and temporal constraints can reduce world-frame errors in single-frame estimates by 40%. Project page: https://egoallo.github.io/ |
| title | Estimating Body and Hand Motion in an Ego-sensed World |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2410.03665 |