HO-Flow: Generalizable Hand-Object Interaction Generation with Latent Flow Matching
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914467388850176 |
|---|---|
| author | Chen, Zerui Potamias, Rolandos Alexandros Chen, Shizhe Deng, Jiankang Schmid, Cordelia Zafeiriou, Stefanos |
| author_facet | Chen, Zerui Potamias, Rolandos Alexandros Chen, Shizhe Deng, Jiankang Schmid, Cordelia Zafeiriou, Stefanos |
| contents | Generating realistic 3D hand-object interactions (HOI) is a fundamental challenge in computer vision and robotics, requiring both temporal coherence and high-fidelity physical plausibility. Existing methods remain limited in their ability to learn expressive motion representations for generation and perform temporal reasoning. In this paper, we present HO-Flow, a framework for synthesizing realistic hand-object motion sequences from texts and canoncial 3D objects. HO-Flow first employs an interaction-aware variational autoencoder to encode sequences of hand and object motions into a unified latent manifold by incorporating hand and object kinematics, enabling the representation to capture rich interaction dynamics. It then leverages a masked flow matching model that combines auto-regressive temporal reasoning with continuous latent generation, improving temporal coherence. To further enhance generalization, HO-Flow predicts object motions relative to the initial frame, enabling effective pre-training on large-scale synthetic data. Experiments on the GRAB, OakInk, and DexYCB benchmarks demonstrate that HO-Flow achieves state-of-the-art performance in both physical plausibility and motion diversity for interaction motion synthesis. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_10836 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | HO-Flow: Generalizable Hand-Object Interaction Generation with Latent Flow Matching Chen, Zerui Potamias, Rolandos Alexandros Chen, Shizhe Deng, Jiankang Schmid, Cordelia Zafeiriou, Stefanos Computer Vision and Pattern Recognition Robotics Generating realistic 3D hand-object interactions (HOI) is a fundamental challenge in computer vision and robotics, requiring both temporal coherence and high-fidelity physical plausibility. Existing methods remain limited in their ability to learn expressive motion representations for generation and perform temporal reasoning. In this paper, we present HO-Flow, a framework for synthesizing realistic hand-object motion sequences from texts and canoncial 3D objects. HO-Flow first employs an interaction-aware variational autoencoder to encode sequences of hand and object motions into a unified latent manifold by incorporating hand and object kinematics, enabling the representation to capture rich interaction dynamics. It then leverages a masked flow matching model that combines auto-regressive temporal reasoning with continuous latent generation, improving temporal coherence. To further enhance generalization, HO-Flow predicts object motions relative to the initial frame, enabling effective pre-training on large-scale synthetic data. Experiments on the GRAB, OakInk, and DexYCB benchmarks demonstrate that HO-Flow achieves state-of-the-art performance in both physical plausibility and motion diversity for interaction motion synthesis. |
| title | HO-Flow: Generalizable Hand-Object Interaction Generation with Latent Flow Matching |
| topic | Computer Vision and Pattern Recognition Robotics |
| url | https://arxiv.org/abs/2604.10836 |