Cover Image

SkyReels-V4: Multi-modal Video-Audio Generation, Inpainting and Editing model

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chen, Guibin, Lin, Dixuan, Yang, Jiangping, Zhang, Youqiang, Fei, Zhengcong, Li, Debang, Chen, Sheng, Ao, Chaofeng, Pang, Nuo, Wang, Yiming, Dou, Yikun, Chen, Zheng, Fan, Mingyuan, Li, Tuanhui, Chang, Mingshan, Zhang, Hao, Sun, Xiaopeng, Xu, Jingtao, Xie, Yuqiang, Wang, Jiahua, Xu, Zhiheng, Xiong, Weiming, Jin, Yuzhe, Gu, Baoxuan, Mao, Binjie, Yu, Yunjie, He, Jujie, Feng, Yuhao, Tu, Shiwen, Wang, Chaojie, Yan, Rui, Shen, Wei, Wu, Jingchen, Zhao, Peng, Zhong, Xuanyue, Liu, Zhuangzhuang, Wang, Kaifei, Zhang, Fuxiang, Xu, Weikai, Liu, Wenyan, Zhang, Binglu, Shen, Yu, Xiong, Tianhui, Peng, Bin, Zeng, Liang, Song, Xuchen, Guo, Haoxiang, Wang, Peiyu, Lam, Max W. Y., Liu, Chien-Hung, Zhou, Yahui
Format:	Preprint
Published:	2026
Subjects:	Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

_version_	1866911524058038272
author	Chen, Guibin Lin, Dixuan Yang, Jiangping Zhang, Youqiang Fei, Zhengcong Li, Debang Chen, Sheng Ao, Chaofeng Pang, Nuo Wang, Yiming Dou, Yikun Chen, Zheng Fan, Mingyuan Li, Tuanhui Chang, Mingshan Zhang, Hao Sun, Xiaopeng Xu, Jingtao Xie, Yuqiang Wang, Jiahua Xu, Zhiheng Xiong, Weiming Jin, Yuzhe Gu, Baoxuan Mao, Binjie Yu, Yunjie He, Jujie Feng, Yuhao Tu, Shiwen Wang, Chaojie Yan, Rui Shen, Wei Wu, Jingchen Zhao, Peng Zhong, Xuanyue Liu, Zhuangzhuang Wang, Kaifei Zhang, Fuxiang Xu, Weikai Liu, Wenyan Zhang, Binglu Shen, Yu Xiong, Tianhui Peng, Bin Zeng, Liang Song, Xuchen Guo, Haoxiang Wang, Peiyu Lam, Max W. Y. Liu, Chien-Hung Zhou, Yahui
author_facet	Chen, Guibin Lin, Dixuan Yang, Jiangping Zhang, Youqiang Fei, Zhengcong Li, Debang Chen, Sheng Ao, Chaofeng Pang, Nuo Wang, Yiming Dou, Yikun Chen, Zheng Fan, Mingyuan Li, Tuanhui Chang, Mingshan Zhang, Hao Sun, Xiaopeng Xu, Jingtao Xie, Yuqiang Wang, Jiahua Xu, Zhiheng Xiong, Weiming Jin, Yuzhe Gu, Baoxuan Mao, Binjie Yu, Yunjie He, Jujie Feng, Yuhao Tu, Shiwen Wang, Chaojie Yan, Rui Shen, Wei Wu, Jingchen Zhao, Peng Zhong, Xuanyue Liu, Zhuangzhuang Wang, Kaifei Zhang, Fuxiang Xu, Weikai Liu, Wenyan Zhang, Binglu Shen, Yu Xiong, Tianhui Peng, Bin Zeng, Liang Song, Xuchen Guo, Haoxiang Wang, Peiyu Lam, Max W. Y. Liu, Chien-Hung Zhou, Yahui
contents	SkyReels V4 is a unified multi modal video foundation model for joint video audio generation, inpainting, and editing. The model adopts a dual stream Multimodal Diffusion Transformer (MMDiT) architecture, where one branch synthesizes video and the other generates temporally aligned audio, while sharing a powerful text encoder based on the Multimodal Large Language Models (MLLM). SkyReels V4 accepts rich multi modal instructions, including text, images, video clips, masks, and audio references. By combining the MLLMs multi modal instruction following capability with in context learning in the video branch MMDiT, the model can inject fine grained visual guidance under complex conditioning, while the audio branch MMDiT simultaneously leverages audio references to guide sound generation. On the video side, we adopt a channel concatenation formulation that unifies a wide range of inpainting style tasks, such as image to video, video extension, and video editing under a single interface, and naturally extends to vision referenced inpainting and editing via multi modal prompts. SkyReels V4 supports up to 1080p resolution, 32 FPS, and 15 second duration, enabling high fidelity, multi shot, cinema level video generation with synchronized audio. To make such high resolution, long-duration generation computationally feasible, we introduce an efficiency strategy: Joint generation of low resolution full sequences and high-resolution keyframes, followed by dedicated super-resolution and frame interpolation models. To our knowledge, SkyReels V4 is the first video foundation model that simultaneously supports multi-modal input, joint video audio generation, and a unified treatment of generation, inpainting, and editing, while maintaining strong efficiency and quality at cinematic resolutions and durations.
format	Preprint
id	arxiv_https___arxiv_org_abs_2602_21818
institution	arXiv
publishDate	2026
record_format	arxiv
spellingShingle	SkyReels-V4: Multi-modal Video-Audio Generation, Inpainting and Editing model Chen, Guibin Lin, Dixuan Yang, Jiangping Zhang, Youqiang Fei, Zhengcong Li, Debang Chen, Sheng Ao, Chaofeng Pang, Nuo Wang, Yiming Dou, Yikun Chen, Zheng Fan, Mingyuan Li, Tuanhui Chang, Mingshan Zhang, Hao Sun, Xiaopeng Xu, Jingtao Xie, Yuqiang Wang, Jiahua Xu, Zhiheng Xiong, Weiming Jin, Yuzhe Gu, Baoxuan Mao, Binjie Yu, Yunjie He, Jujie Feng, Yuhao Tu, Shiwen Wang, Chaojie Yan, Rui Shen, Wei Wu, Jingchen Zhao, Peng Zhong, Xuanyue Liu, Zhuangzhuang Wang, Kaifei Zhang, Fuxiang Xu, Weikai Liu, Wenyan Zhang, Binglu Shen, Yu Xiong, Tianhui Peng, Bin Zeng, Liang Song, Xuchen Guo, Haoxiang Wang, Peiyu Lam, Max W. Y. Liu, Chien-Hung Zhou, Yahui Computer Vision and Pattern Recognition SkyReels V4 is a unified multi modal video foundation model for joint video audio generation, inpainting, and editing. The model adopts a dual stream Multimodal Diffusion Transformer (MMDiT) architecture, where one branch synthesizes video and the other generates temporally aligned audio, while sharing a powerful text encoder based on the Multimodal Large Language Models (MLLM). SkyReels V4 accepts rich multi modal instructions, including text, images, video clips, masks, and audio references. By combining the MLLMs multi modal instruction following capability with in context learning in the video branch MMDiT, the model can inject fine grained visual guidance under complex conditioning, while the audio branch MMDiT simultaneously leverages audio references to guide sound generation. On the video side, we adopt a channel concatenation formulation that unifies a wide range of inpainting style tasks, such as image to video, video extension, and video editing under a single interface, and naturally extends to vision referenced inpainting and editing via multi modal prompts. SkyReels V4 supports up to 1080p resolution, 32 FPS, and 15 second duration, enabling high fidelity, multi shot, cinema level video generation with synchronized audio. To make such high resolution, long-duration generation computationally feasible, we introduce an efficiency strategy: Joint generation of low resolution full sequences and high-resolution keyframes, followed by dedicated super-resolution and frame interpolation models. To our knowledge, SkyReels V4 is the first video foundation model that simultaneously supports multi-modal input, joint video audio generation, and a unified treatment of generation, inpainting, and editing, while maintaining strong efficiency and quality at cinematic resolutions and durations.
title	SkyReels-V4: Multi-modal Video-Audio Generation, Inpainting and Editing model
topic	Computer Vision and Pattern Recognition
url	https://arxiv.org/abs/2602.21818