Saved in:
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2412.10255 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916750541455360 |
|---|---|
| author | Jiang, Yudong Xu, Baohan Yang, Siqian Yin, Mingyu Liu, Jing Xu, Chao Wang, Siqi Wu, Yidi Zhu, Bingwen Zhang, Xinwen Zheng, Xingyu Xu, Jixuan Zhang, Yue Hou, Jinlong Sun, Huyang |
| author_facet | Jiang, Yudong Xu, Baohan Yang, Siqian Yin, Mingyu Liu, Jing Xu, Chao Wang, Siqi Wu, Yidi Zhu, Bingwen Zhang, Xinwen Zheng, Xingyu Xu, Jixuan Zhang, Yue Hou, Jinlong Sun, Huyang |
| contents | Animation has gained significant interest in the recent film and TV industry. Despite the success of advanced video generation models like Sora, Kling, and CogVideoX in generating natural videos, they lack the same effectiveness in handling animation videos. Evaluating animation video generation is also a great challenge due to its unique artist styles, violating the laws of physics and exaggerated motions. In this paper, we present a comprehensive system, AniSora, designed for animation video generation, which includes a data processing pipeline, a controllable generation model, and an evaluation benchmark. Supported by the data processing pipeline with over 10M high-quality data, the generation model incorporates a spatiotemporal mask module to facilitate key animation production functions such as image-to-video generation, frame interpolation, and localized image-guided animation. We also collect an evaluation benchmark of 948 various animation videos, with specifically developed metrics for animation video generation. Our entire project is publicly available on https://github.com/bilibili/Index-anisora/tree/main. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_10255 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | AniSora: Exploring the Frontiers of Animation Video Generation in the Sora Era Jiang, Yudong Xu, Baohan Yang, Siqian Yin, Mingyu Liu, Jing Xu, Chao Wang, Siqi Wu, Yidi Zhu, Bingwen Zhang, Xinwen Zheng, Xingyu Xu, Jixuan Zhang, Yue Hou, Jinlong Sun, Huyang Graphics Artificial Intelligence Animation has gained significant interest in the recent film and TV industry. Despite the success of advanced video generation models like Sora, Kling, and CogVideoX in generating natural videos, they lack the same effectiveness in handling animation videos. Evaluating animation video generation is also a great challenge due to its unique artist styles, violating the laws of physics and exaggerated motions. In this paper, we present a comprehensive system, AniSora, designed for animation video generation, which includes a data processing pipeline, a controllable generation model, and an evaluation benchmark. Supported by the data processing pipeline with over 10M high-quality data, the generation model incorporates a spatiotemporal mask module to facilitate key animation production functions such as image-to-video generation, frame interpolation, and localized image-guided animation. We also collect an evaluation benchmark of 948 various animation videos, with specifically developed metrics for animation video generation. Our entire project is publicly available on https://github.com/bilibili/Index-anisora/tree/main. |
| title | AniSora: Exploring the Frontiers of Animation Video Generation in the Sora Era |
| topic | Graphics Artificial Intelligence |
| url | https://arxiv.org/abs/2412.10255 |