MANUS: Markerless Grasp Capture using Articulated 3D Gaussians
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929293162971136 |
|---|---|
| author | Pokhariya, Chandradeep Shah, Ishaan N Xing, Angela Li, Zekun Chen, Kefan Sharma, Avinash Sridhar, Srinath |
| author_facet | Pokhariya, Chandradeep Shah, Ishaan N Xing, Angela Li, Zekun Chen, Kefan Sharma, Avinash Sridhar, Srinath |
| contents | Understanding how we grasp objects with our hands has important applications in areas like robotics and mixed reality. However, this challenging problem requires accurate modeling of the contact between hands and objects. To capture grasps, existing methods use skeletons, meshes, or parametric models that does not represent hand shape accurately resulting in inaccurate contacts. We present MANUS, a method for Markerless Hand-Object Grasp Capture using Articulated 3D Gaussians. We build a novel articulated 3D Gaussians representation that extends 3D Gaussian splatting for high-fidelity representation of articulating hands. Since our representation uses Gaussian primitives, it enables us to efficiently and accurately estimate contacts between the hand and the object. For the most accurate results, our method requires tens of camera views that current datasets do not provide. We therefore build MANUS-Grasps, a new dataset that contains hand-object grasps viewed from 50+ cameras across 30+ scenes, 3 subjects, and comprising over 7M frames. In addition to extensive qualitative results, we also show that our method outperforms others on a quantitative contact evaluation method that uses paint transfer from the object to the hand. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_02137 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | MANUS: Markerless Grasp Capture using Articulated 3D Gaussians Pokhariya, Chandradeep Shah, Ishaan N Xing, Angela Li, Zekun Chen, Kefan Sharma, Avinash Sridhar, Srinath Computer Vision and Pattern Recognition Understanding how we grasp objects with our hands has important applications in areas like robotics and mixed reality. However, this challenging problem requires accurate modeling of the contact between hands and objects. To capture grasps, existing methods use skeletons, meshes, or parametric models that does not represent hand shape accurately resulting in inaccurate contacts. We present MANUS, a method for Markerless Hand-Object Grasp Capture using Articulated 3D Gaussians. We build a novel articulated 3D Gaussians representation that extends 3D Gaussian splatting for high-fidelity representation of articulating hands. Since our representation uses Gaussian primitives, it enables us to efficiently and accurately estimate contacts between the hand and the object. For the most accurate results, our method requires tens of camera views that current datasets do not provide. We therefore build MANUS-Grasps, a new dataset that contains hand-object grasps viewed from 50+ cameras across 30+ scenes, 3 subjects, and comprising over 7M frames. In addition to extensive qualitative results, we also show that our method outperforms others on a quantitative contact evaluation method that uses paint transfer from the object to the hand. |
| title | MANUS: Markerless Grasp Capture using Articulated 3D Gaussians |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2312.02137 |