Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.00960 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915873277607936 |
|---|---|
| author | Wen, Boran Lu, Ye Wang, Sirui Wan, Keyan Zhou, Jiahong Liang, Junxuan Liu, Xinpeng Xiao, Bang Liu, Ruiyang Li, Yong-Lu |
| author_facet | Wen, Boran Lu, Ye Wang, Sirui Wan, Keyan Zhou, Jiahong Liang, Junxuan Liu, Xinpeng Xiao, Bang Liu, Ruiyang Li, Yong-Lu |
| contents | Generalized robots must learn from diverse, large-scale human-object interactions (HOI) to operate robustly in the real world. Monocular internet videos offer a nearly limitless and readily available source of data, capturing an unparalleled diversity of human activities, objects, and environments. However, accurately and scalably extracting 4D interaction data from these in-the-wild videos remains a significant and unsolved challenge. To overcome the annotation bottleneck, we introduce an efficient sparse contact annotation paradigm. To scale this process, we develop InterPoint, a multi-modal predictor that drives a human-in-the-loop data engine. Building upon these efficiently acquired annotations, we introduce 4DHOISolver, a novel optimization framework that constrains the ill-posed 4D HOI reconstruction problem, maintaining high spatio-temporal coherence and physical plausibility. Leveraging this framework, we introduce Open4DHOI, a new large-scale 4D HOI dataset featuring a diverse catalog of 135 object types and 133 actions. Furthermore, we demonstrate the effectiveness of our reconstructions by enabling an RL-based agent to imitate the recovered motions. Data and code will be publicly available at https://github.com/wenboran2002/open4dhoi_code. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_00960 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Efficient and Scalable Monocular Human-Object Interaction Motion Reconstruction Wen, Boran Lu, Ye Wang, Sirui Wan, Keyan Zhou, Jiahong Liang, Junxuan Liu, Xinpeng Xiao, Bang Liu, Ruiyang Li, Yong-Lu Computer Vision and Pattern Recognition Generalized robots must learn from diverse, large-scale human-object interactions (HOI) to operate robustly in the real world. Monocular internet videos offer a nearly limitless and readily available source of data, capturing an unparalleled diversity of human activities, objects, and environments. However, accurately and scalably extracting 4D interaction data from these in-the-wild videos remains a significant and unsolved challenge. To overcome the annotation bottleneck, we introduce an efficient sparse contact annotation paradigm. To scale this process, we develop InterPoint, a multi-modal predictor that drives a human-in-the-loop data engine. Building upon these efficiently acquired annotations, we introduce 4DHOISolver, a novel optimization framework that constrains the ill-posed 4D HOI reconstruction problem, maintaining high spatio-temporal coherence and physical plausibility. Leveraging this framework, we introduce Open4DHOI, a new large-scale 4D HOI dataset featuring a diverse catalog of 135 object types and 133 actions. Furthermore, we demonstrate the effectiveness of our reconstructions by enabling an RL-based agent to imitate the recovered motions. Data and code will be publicly available at https://github.com/wenboran2002/open4dhoi_code. |
| title | Efficient and Scalable Monocular Human-Object Interaction Motion Reconstruction |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2512.00960 |