Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2023
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2309.07891 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910598339493888 |
|---|---|
| author | Choi, Hongsuk Chavan-Dafle, Nikhil Yuan, Jiacheng Isler, Volkan Park, Hyunsoo |
| author_facet | Choi, Hongsuk Chavan-Dafle, Nikhil Yuan, Jiacheng Isler, Volkan Park, Hyunsoo |
| contents | This paper presents a method to learn hand-object interaction prior for reconstructing a 3D hand-object scene from a single RGB image. The inference as well as training-data generation for 3D hand-object scene reconstruction is challenging due to the depth ambiguity of a single image and occlusions by the hand and object. We turn this challenge into an opportunity by utilizing the hand shape to constrain the possible relative configuration of the hand and object geometry. We design a generalizable implicit function, HandNeRF, that explicitly encodes the correlation of the 3D hand shape features and 2D object features to predict the hand and object scene geometry. With experiments on real-world datasets, we show that HandNeRF is able to reconstruct hand-object scenes of novel grasp configurations more accurately than comparable methods. Moreover, we demonstrate that object reconstruction from HandNeRF ensures more accurate execution of downstream tasks, such as grasping and motion planning for robotic hand-over and manipulation. Homepage: https://samsunglabs.github.io/HandNeRF-project-page/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_07891 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | HandNeRF: Learning to Reconstruct Hand-Object Interaction Scene from a Single RGB Image Choi, Hongsuk Chavan-Dafle, Nikhil Yuan, Jiacheng Isler, Volkan Park, Hyunsoo Computer Vision and Pattern Recognition This paper presents a method to learn hand-object interaction prior for reconstructing a 3D hand-object scene from a single RGB image. The inference as well as training-data generation for 3D hand-object scene reconstruction is challenging due to the depth ambiguity of a single image and occlusions by the hand and object. We turn this challenge into an opportunity by utilizing the hand shape to constrain the possible relative configuration of the hand and object geometry. We design a generalizable implicit function, HandNeRF, that explicitly encodes the correlation of the 3D hand shape features and 2D object features to predict the hand and object scene geometry. With experiments on real-world datasets, we show that HandNeRF is able to reconstruct hand-object scenes of novel grasp configurations more accurately than comparable methods. Moreover, we demonstrate that object reconstruction from HandNeRF ensures more accurate execution of downstream tasks, such as grasping and motion planning for robotic hand-over and manipulation. Homepage: https://samsunglabs.github.io/HandNeRF-project-page/ |
| title | HandNeRF: Learning to Reconstruct Hand-Object Interaction Scene from a Single RGB Image |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2309.07891 |