Towards Multiple Character Image Animation Through Enhancing Implicit Decoupling
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909518647001088 |
|---|---|
| author | Xue, Jingyun Wang, Hongfa Tian, Qi Ma, Yue Wang, Andong Zhao, Zhiyuan Min, Shaobo Zhao, Wenzhe Zhang, Kaihao Shum, Heung-Yeung Liu, Wei Liu, Mengyang Luo, Wenhan |
| author_facet | Xue, Jingyun Wang, Hongfa Tian, Qi Ma, Yue Wang, Andong Zhao, Zhiyuan Min, Shaobo Zhao, Wenzhe Zhang, Kaihao Shum, Heung-Yeung Liu, Wei Liu, Mengyang Luo, Wenhan |
| contents | Controllable character image animation has a wide range of applications. Although existing studies have consistently improved performance, challenges persist in the field of character image animation, particularly concerning stability in complex backgrounds and tasks involving multiple characters. To address these challenges, we propose a novel multi-condition guided framework for character image animation, employing several well-designed input modules to enhance the implicit decoupling capability of the model. First, the optical flow guider calculates the background optical flow map as guidance information, which enables the model to implicitly learn to decouple the background motion into background constants and background momentum during training, and generate a stable background by setting zero background momentum during inference. Second, the depth order guider calculates the order map of the characters, which transforms the depth information into the positional information of multiple characters. This facilitates the implicit learning of decoupling different characters, especially in accurately separating the occluded body parts of multiple characters. Third, the reference pose map is input to enhance the ability to decouple character texture and pose information in the reference image. Furthermore, to fill the gap of fair evaluation of multi-character image animation, we propose a new benchmark comprising about 4,000 frames. Extensive qualitative and quantitative evaluations demonstrate that our method excels in generating high-quality character animations, especially in scenarios of complex backgrounds and multiple characters. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_03035 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Towards Multiple Character Image Animation Through Enhancing Implicit Decoupling Xue, Jingyun Wang, Hongfa Tian, Qi Ma, Yue Wang, Andong Zhao, Zhiyuan Min, Shaobo Zhao, Wenzhe Zhang, Kaihao Shum, Heung-Yeung Liu, Wei Liu, Mengyang Luo, Wenhan Computer Vision and Pattern Recognition Controllable character image animation has a wide range of applications. Although existing studies have consistently improved performance, challenges persist in the field of character image animation, particularly concerning stability in complex backgrounds and tasks involving multiple characters. To address these challenges, we propose a novel multi-condition guided framework for character image animation, employing several well-designed input modules to enhance the implicit decoupling capability of the model. First, the optical flow guider calculates the background optical flow map as guidance information, which enables the model to implicitly learn to decouple the background motion into background constants and background momentum during training, and generate a stable background by setting zero background momentum during inference. Second, the depth order guider calculates the order map of the characters, which transforms the depth information into the positional information of multiple characters. This facilitates the implicit learning of decoupling different characters, especially in accurately separating the occluded body parts of multiple characters. Third, the reference pose map is input to enhance the ability to decouple character texture and pose information in the reference image. Furthermore, to fill the gap of fair evaluation of multi-character image animation, we propose a new benchmark comprising about 4,000 frames. Extensive qualitative and quantitative evaluations demonstrate that our method excels in generating high-quality character animations, especially in scenarios of complex backgrounds and multiple characters. |
| title | Towards Multiple Character Image Animation Through Enhancing Implicit Decoupling |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2406.03035 |