MVHOI: Bridge Multi-view Condition to Complex Human-Object Interaction Video Reenactment via 3D Foundation Model
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911518582374400 |
|---|---|
| author | Tong, Jinguang Wu, Jinbo Wang, Kaisiyuan Shen, Zhelun Huang, Xuan Xiang, Mochu Li, Xuesong Li, Yingying Feng, Haocheng Zhao, Chen Zhou, Hang He, Wei Nguyen, Chuong Wang, Jingdong Li, Hongdong |
| author_facet | Tong, Jinguang Wu, Jinbo Wang, Kaisiyuan Shen, Zhelun Huang, Xuan Xiang, Mochu Li, Xuesong Li, Yingying Feng, Haocheng Zhao, Chen Zhou, Hang He, Wei Nguyen, Chuong Wang, Jingdong Li, Hongdong |
| contents | Human-Object Interaction (HOI) video reenactment with realistic motion remains a frontier in expressive digital human creation. Existing approaches primarily handle simple image-plane motion (e.g., in-plane translations), struggling with complex non-planar manipulations like out-of-plane reorientation. In this paper, we propose MVHOI, a two-stage HOI video reenactment framework that bridges multi-view reference conditions and video foundation models via a 3D Foundation Model (3DFM). The 3DFM first produces view-consistent object priors conditioned on implicit motion dynamics across novel viewpoints. A controllable video generation model then synthesizes high-fidelity object texture by incorporating multi-view reference images, ensuring appearance consistency via a reasonable retrieval mechanism. By enabling these two stages to mutually reinforce one another during the inference phase, our framework shows superior performance in generating long-duration HOI videos with intricate object manipulations. Extensive experiments show substantial improvements over prior approaches, especially for HOI with complex 3D object manipulations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_14686 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | MVHOI: Bridge Multi-view Condition to Complex Human-Object Interaction Video Reenactment via 3D Foundation Model Tong, Jinguang Wu, Jinbo Wang, Kaisiyuan Shen, Zhelun Huang, Xuan Xiang, Mochu Li, Xuesong Li, Yingying Feng, Haocheng Zhao, Chen Zhou, Hang He, Wei Nguyen, Chuong Wang, Jingdong Li, Hongdong Computer Vision and Pattern Recognition Artificial Intelligence Human-Object Interaction (HOI) video reenactment with realistic motion remains a frontier in expressive digital human creation. Existing approaches primarily handle simple image-plane motion (e.g., in-plane translations), struggling with complex non-planar manipulations like out-of-plane reorientation. In this paper, we propose MVHOI, a two-stage HOI video reenactment framework that bridges multi-view reference conditions and video foundation models via a 3D Foundation Model (3DFM). The 3DFM first produces view-consistent object priors conditioned on implicit motion dynamics across novel viewpoints. A controllable video generation model then synthesizes high-fidelity object texture by incorporating multi-view reference images, ensuring appearance consistency via a reasonable retrieval mechanism. By enabling these two stages to mutually reinforce one another during the inference phase, our framework shows superior performance in generating long-duration HOI videos with intricate object manipulations. Extensive experiments show substantial improvements over prior approaches, especially for HOI with complex 3D object manipulations. |
| title | MVHOI: Bridge Multi-view Condition to Complex Human-Object Interaction Video Reenactment via 3D Foundation Model |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2603.14686 |