Saved in:
| Main Authors: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2602.08682 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911438646280192 |
|---|---|
| author | Guo, Ying Gan, Qijun Zhang, Yifu Liu, Jinlai Hu, Yifei Xie, Pan Qian, Dongjun Zhang, Yu Li, Ruiqi Zhang, Yuqi Lu, Ruibiao Mei, Xiaofeng Han, Bo Yin, Xiang Peng, Bingyue Yuan, Zehuan |
| author_facet | Guo, Ying Gan, Qijun Zhang, Yifu Liu, Jinlai Hu, Yifei Xie, Pan Qian, Dongjun Zhang, Yu Li, Ruiqi Zhang, Yuqi Lu, Ruibiao Mei, Xiaofeng Han, Bo Yin, Xiang Peng, Bingyue Yuan, Zehuan |
| contents | Video generation is rapidly evolving towards unified audio-video generation. In this paper, we present ALIVE, a generation model that adapts a pretrained Text-to-Video (T2V) model to Sora-style audio-video generation and animation. In particular, the model unlocks the Text-to-Video&Audio (T2VA) and Reference-to-Video&Audio (animation) capabilities compared to the T2V foundation models. To support the audio-visual synchronization and reference animation, we augment the popular MMDiT architecture with a joint audio-video branch which includes TA-CrossAttn for temporally-aligned cross-modal fusion and UniTemp-RoPE for precise audio-visual alignment. Meanwhile, a comprehensive data pipeline consisting of audio-video captioning, quality control, etc., is carefully designed to collect high-quality finetuning data. Additionally, we introduce a new benchmark to perform a comprehensive model test and comparison. After continue pretraining and finetuning on million-level high-quality data, ALIVE demonstrates outstanding performance, consistently outperforming open-source models and matching or surpassing state-of-the-art commercial solutions. With detailed recipes and benchmarks, we hope ALIVE helps the community develop audio-video generation models more efficiently. Official page: https://github.com/FoundationVision/Alive. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_08682 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | ALIVE: Animate Your World with Lifelike Audio-Video Generation Guo, Ying Gan, Qijun Zhang, Yifu Liu, Jinlai Hu, Yifei Xie, Pan Qian, Dongjun Zhang, Yu Li, Ruiqi Zhang, Yuqi Lu, Ruibiao Mei, Xiaofeng Han, Bo Yin, Xiang Peng, Bingyue Yuan, Zehuan Computer Vision and Pattern Recognition Video generation is rapidly evolving towards unified audio-video generation. In this paper, we present ALIVE, a generation model that adapts a pretrained Text-to-Video (T2V) model to Sora-style audio-video generation and animation. In particular, the model unlocks the Text-to-Video&Audio (T2VA) and Reference-to-Video&Audio (animation) capabilities compared to the T2V foundation models. To support the audio-visual synchronization and reference animation, we augment the popular MMDiT architecture with a joint audio-video branch which includes TA-CrossAttn for temporally-aligned cross-modal fusion and UniTemp-RoPE for precise audio-visual alignment. Meanwhile, a comprehensive data pipeline consisting of audio-video captioning, quality control, etc., is carefully designed to collect high-quality finetuning data. Additionally, we introduce a new benchmark to perform a comprehensive model test and comparison. After continue pretraining and finetuning on million-level high-quality data, ALIVE demonstrates outstanding performance, consistently outperforming open-source models and matching or surpassing state-of-the-art commercial solutions. With detailed recipes and benchmarks, we hope ALIVE helps the community develop audio-video generation models more efficiently. Official page: https://github.com/FoundationVision/Alive. |
| title | ALIVE: Animate Your World with Lifelike Audio-Video Generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2602.08682 |