Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2411.17383 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915354245070848 |
|---|---|
| author | Xu, Ziyi Huang, Ziyao Cao, Juan Zhang, Yong Cun, Xiaodong Shuai, Qing Wang, Yuchen Bao, Linchao Li, Jintao Tang, Fan |
| author_facet | Xu, Ziyi Huang, Ziyao Cao, Juan Zhang, Yong Cun, Xiaodong Shuai, Qing Wang, Yuchen Bao, Linchao Li, Jintao Tang, Fan |
| contents | The generation of anchor-style product promotion videos presents promising opportunities in e-commerce, advertising, and consumer engagement. Despite advancements in pose-guided human video generation, creating product promotion videos remains challenging. In addressing this challenge, we identify the integration of human-object interactions (HOI) into pose-guided human video generation as a core issue. To this end, we introduce AnchorCrafter, a novel diffusion-based system designed to generate 2D videos featuring a target human and a customized object, achieving high visual fidelity and controllable interactions. Specifically, we propose two key innovations: the HOI-appearance perception, which enhances object appearance recognition from arbitrary multi-view perspectives and disentangles object and human appearance, and the HOI-motion injection, which enables complex human-object interactions by overcoming challenges in object trajectory conditioning and inter-occlusion management. Extensive experiments show that our system improves object appearance preservation by 7.5\% and doubles the object localization accuracy compared to existing state-of-the-art approaches. It also outperforms existing approaches in maintaining human motion consistency and high-quality video generation. Project page including data, code, and Huggingface demo: https://github.com/cangcz/AnchorCrafter. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_17383 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | AnchorCrafter: Animate Cyber-Anchors Selling Your Products via Human-Object Interacting Video Generation Xu, Ziyi Huang, Ziyao Cao, Juan Zhang, Yong Cun, Xiaodong Shuai, Qing Wang, Yuchen Bao, Linchao Li, Jintao Tang, Fan Computer Vision and Pattern Recognition The generation of anchor-style product promotion videos presents promising opportunities in e-commerce, advertising, and consumer engagement. Despite advancements in pose-guided human video generation, creating product promotion videos remains challenging. In addressing this challenge, we identify the integration of human-object interactions (HOI) into pose-guided human video generation as a core issue. To this end, we introduce AnchorCrafter, a novel diffusion-based system designed to generate 2D videos featuring a target human and a customized object, achieving high visual fidelity and controllable interactions. Specifically, we propose two key innovations: the HOI-appearance perception, which enhances object appearance recognition from arbitrary multi-view perspectives and disentangles object and human appearance, and the HOI-motion injection, which enables complex human-object interactions by overcoming challenges in object trajectory conditioning and inter-occlusion management. Extensive experiments show that our system improves object appearance preservation by 7.5\% and doubles the object localization accuracy compared to existing state-of-the-art approaches. It also outperforms existing approaches in maintaining human motion consistency and high-quality video generation. Project page including data, code, and Huggingface demo: https://github.com/cangcz/AnchorCrafter. |
| title | AnchorCrafter: Animate Cyber-Anchors Selling Your Products via Human-Object Interacting Video Generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2411.17383 |