Depth Restoration of Hand-Held Transparent Objects for Human-to-Robot Handover
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914949832376320 |
|---|---|
| author | Yu, Ran Yu, Haixin Li, Shoujie Yan, Huang Song, Ziwu Ding, Wenbo |
| author_facet | Yu, Ran Yu, Haixin Li, Shoujie Yan, Huang Song, Ziwu Ding, Wenbo |
| contents | Transparent objects are common in daily life, while their optical properties pose challenges for RGB-D cameras to capture accurate depth information. This issue is further amplified when these objects are hand-held, as hand occlusions further complicate depth estimation. For assistant robots, however, accurately perceiving hand-held transparent objects is critical to effective human-robot interaction. This paper presents a Hand-Aware Depth Restoration (HADR) method based on creating an implicit neural representation function from a single RGB-D image. The proposed method utilizes hand posture as an important guidance to leverage semantic and geometric information of hand-object interaction. To train and evaluate the proposed method, we create a high-fidelity synthetic dataset named TransHand-14K with a real-to-sim data generation scheme. Experiments show that our method has better performance and generalization ability compared with existing methods. We further develop a real-world human-to-robot handover system based on HADR, demonstrating its potential in human-robot interaction applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_14997 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Depth Restoration of Hand-Held Transparent Objects for Human-to-Robot Handover Yu, Ran Yu, Haixin Li, Shoujie Yan, Huang Song, Ziwu Ding, Wenbo Robotics Computer Vision and Pattern Recognition Transparent objects are common in daily life, while their optical properties pose challenges for RGB-D cameras to capture accurate depth information. This issue is further amplified when these objects are hand-held, as hand occlusions further complicate depth estimation. For assistant robots, however, accurately perceiving hand-held transparent objects is critical to effective human-robot interaction. This paper presents a Hand-Aware Depth Restoration (HADR) method based on creating an implicit neural representation function from a single RGB-D image. The proposed method utilizes hand posture as an important guidance to leverage semantic and geometric information of hand-object interaction. To train and evaluate the proposed method, we create a high-fidelity synthetic dataset named TransHand-14K with a real-to-sim data generation scheme. Experiments show that our method has better performance and generalization ability compared with existing methods. We further develop a real-world human-to-robot handover system based on HADR, demonstrating its potential in human-robot interaction applications. |
| title | Depth Restoration of Hand-Held Transparent Objects for Human-to-Robot Handover |
| topic | Robotics Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2408.14997 |