Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2405.02676 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914784058802176 |
|---|---|
| author | Hu, Haoyu Yi, Xinyu Cao, Zhe Yong, Jun-Hai Xu, Feng |
| author_facet | Hu, Haoyu Yi, Xinyu Cao, Zhe Yong, Jun-Hai Xu, Feng |
| contents | Hand manipulating objects is an important interaction motion in our daily activities. We faithfully reconstruct this motion with a single RGBD camera by a novel deep reinforcement learning method to leverage physics. Firstly, we propose object compensation control which establishes direct object control to make the network training more stable. Meanwhile, by leveraging the compensation force and torque, we seamlessly upgrade the simple point contact model to a more physical-plausible surface contact model, further improving the reconstruction accuracy and physical correctness. Experiments indicate that without involving any heuristic physical rules, this work still successfully involves physics in the reconstruction of hand-object interactions which are complex motions hard to imitate with deep reinforcement learning. Our code and data are available at https://github.com/hu-hy17/HOIC. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_02676 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Hand-Object Interaction Controller (HOIC): Deep Reinforcement Learning for Reconstructing Interactions with Physics Hu, Haoyu Yi, Xinyu Cao, Zhe Yong, Jun-Hai Xu, Feng Computer Vision and Pattern Recognition Graphics I.5.4 Hand manipulating objects is an important interaction motion in our daily activities. We faithfully reconstruct this motion with a single RGBD camera by a novel deep reinforcement learning method to leverage physics. Firstly, we propose object compensation control which establishes direct object control to make the network training more stable. Meanwhile, by leveraging the compensation force and torque, we seamlessly upgrade the simple point contact model to a more physical-plausible surface contact model, further improving the reconstruction accuracy and physical correctness. Experiments indicate that without involving any heuristic physical rules, this work still successfully involves physics in the reconstruction of hand-object interactions which are complex motions hard to imitate with deep reinforcement learning. Our code and data are available at https://github.com/hu-hy17/HOIC. |
| title | Hand-Object Interaction Controller (HOIC): Deep Reinforcement Learning for Reconstructing Interactions with Physics |
| topic | Computer Vision and Pattern Recognition Graphics I.5.4 |
| url | https://arxiv.org/abs/2405.02676 |