Glove2Hand: Synthesizing Natural Hand-Object Interaction from Multi-Modal Sensing Gloves
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910062585315328 |
|---|---|
| author | Zhang, Xinyu Kou, Ziyi Qin, Chuan Huang, Mia Ristani, Ergys Kumar, Ankit Chen, Lele He, Kun Boularias, Abdeslam Guan, Li |
| author_facet | Zhang, Xinyu Kou, Ziyi Qin, Chuan Huang, Mia Ristani, Ergys Kumar, Ankit Chen, Lele He, Kun Boularias, Abdeslam Guan, Li |
| contents | Understanding hand-object interaction (HOI) is fundamental to computer vision, robotics, and AR/VR. However, conventional hand videos often lack essential physical information such as contact forces and motion signals, and are prone to frequent occlusions. To address the challenges, we present Glove2Hand, a framework that translates multi-modal sensing glove HOI videos into photorealistic bare hands, while faithfully preserving the underlying physical interaction dynamics. We introduce a novel 3D Gaussian hand model that ensures temporal rendering consistency. The rendered hand is seamlessly integrated into the scene using a diffusion-based hand restorer, which effectively handles complex hand-object interactions and non-rigid deformations. Leveraging Glove2Hand, we create HandSense, the first multi-modal HOI dataset featuring glove-to-hand videos with synchronized tactile and IMU signals. We demonstrate that HandSense significantly enhances downstream bare-hand applications, including video-based contact estimation and hand tracking under severe occlusion. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_20850 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Glove2Hand: Synthesizing Natural Hand-Object Interaction from Multi-Modal Sensing Gloves Zhang, Xinyu Kou, Ziyi Qin, Chuan Huang, Mia Ristani, Ergys Kumar, Ankit Chen, Lele He, Kun Boularias, Abdeslam Guan, Li Computer Vision and Pattern Recognition Robotics Understanding hand-object interaction (HOI) is fundamental to computer vision, robotics, and AR/VR. However, conventional hand videos often lack essential physical information such as contact forces and motion signals, and are prone to frequent occlusions. To address the challenges, we present Glove2Hand, a framework that translates multi-modal sensing glove HOI videos into photorealistic bare hands, while faithfully preserving the underlying physical interaction dynamics. We introduce a novel 3D Gaussian hand model that ensures temporal rendering consistency. The rendered hand is seamlessly integrated into the scene using a diffusion-based hand restorer, which effectively handles complex hand-object interactions and non-rigid deformations. Leveraging Glove2Hand, we create HandSense, the first multi-modal HOI dataset featuring glove-to-hand videos with synchronized tactile and IMU signals. We demonstrate that HandSense significantly enhances downstream bare-hand applications, including video-based contact estimation and hand tracking under severe occlusion. |
| title | Glove2Hand: Synthesizing Natural Hand-Object Interaction from Multi-Modal Sensing Gloves |
| topic | Computer Vision and Pattern Recognition Robotics |
| url | https://arxiv.org/abs/2603.20850 |