Guardado en:
| Autores principales: | , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2604.11804 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908973028868096 |
|---|---|
| author | Zhou, Donghao Liu, Guisheng Yang, Hao Li, Jiatong Lin, Jingyu Huang, Xiaohu Liu, Yichen Gao, Xin Chen, Cunjian Wen, Shilei Fu, Chi-Wing Heng, Pheng-Ann |
| author_facet | Zhou, Donghao Liu, Guisheng Yang, Hao Li, Jiatong Lin, Jingyu Huang, Xiaohu Liu, Yichen Gao, Xin Chen, Cunjian Wen, Shilei Fu, Chi-Wing Heng, Pheng-Ann |
| contents | In this work, we study Human-Object Interaction Video Generation (HOIVG), which aims to synthesize high-quality human-object interaction videos conditioned on text, reference images, audio, and pose. This task holds significant practical value for automating content creation in real-world applications, such as e-commerce demonstrations, short video production, and interactive entertainment. However, existing approaches fail to accommodate all these requisite conditions. We present OmniShow, an end-to-end framework tailored for this practical yet challenging task, capable of harmonizing multimodal conditions and delivering industry-grade performance. To overcome the trade-off between controllability and quality, we introduce Unified Channel-wise Conditioning for efficient image and pose injection, and Gated Local-Context Attention to ensure precise audio-visual synchronization. To effectively address data scarcity, we develop a Decoupled-Then-Joint Training strategy that leverages a multi-stage training process with model merging to efficiently harness heterogeneous sub-task datasets. Furthermore, to fill the evaluation gap in this field, we establish HOIVG-Bench, a dedicated and comprehensive benchmark for HOIVG. Extensive experiments demonstrate that OmniShow achieves overall state-of-the-art performance across various multimodal conditioning settings, setting a solid standard for the emerging HOIVG task. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_11804 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | OmniShow: Unifying Multimodal Conditions for Human-Object Interaction Video Generation Zhou, Donghao Liu, Guisheng Yang, Hao Li, Jiatong Lin, Jingyu Huang, Xiaohu Liu, Yichen Gao, Xin Chen, Cunjian Wen, Shilei Fu, Chi-Wing Heng, Pheng-Ann Computer Vision and Pattern Recognition In this work, we study Human-Object Interaction Video Generation (HOIVG), which aims to synthesize high-quality human-object interaction videos conditioned on text, reference images, audio, and pose. This task holds significant practical value for automating content creation in real-world applications, such as e-commerce demonstrations, short video production, and interactive entertainment. However, existing approaches fail to accommodate all these requisite conditions. We present OmniShow, an end-to-end framework tailored for this practical yet challenging task, capable of harmonizing multimodal conditions and delivering industry-grade performance. To overcome the trade-off between controllability and quality, we introduce Unified Channel-wise Conditioning for efficient image and pose injection, and Gated Local-Context Attention to ensure precise audio-visual synchronization. To effectively address data scarcity, we develop a Decoupled-Then-Joint Training strategy that leverages a multi-stage training process with model merging to efficiently harness heterogeneous sub-task datasets. Furthermore, to fill the evaluation gap in this field, we establish HOIVG-Bench, a dedicated and comprehensive benchmark for HOIVG. Extensive experiments demonstrate that OmniShow achieves overall state-of-the-art performance across various multimodal conditioning settings, setting a solid standard for the emerging HOIVG task. |
| title | OmniShow: Unifying Multimodal Conditions for Human-Object Interaction Video Generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2604.11804 |