Cover Image

SkyReels-Audio: Omni Audio-Conditioned Talking Portraits in Video Diffusion Transformers

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Fei, Zhengcong, Jiang, Hao, Qiu, Di, Gu, Baoxuan, Zhang, Youqiang, Wang, Jiahua, Bai, Jialin, Li, Debang, Fan, Mingyuan, Chen, Guibin, Zhou, Yahui
Format:	Preprint
Published:	2025
Subjects:	Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SkyReels-A1: Expressive Portrait Animation in Video Diffusion Transformers
by: Qiu, Di, et al.
Published: (2025)

SkyReels-A2: Compose Anything in Video Diffusion Transformers
by: Fei, Zhengcong, et al.
Published: (2025)

SkyReels-V3 Technique Report
by: Li, Debang, et al.
Published: (2026)

SkyReels-V4: Multi-modal Video-Audio Generation, Inpainting and Editing model
by: Chen, Guibin, et al.
Published: (2026)

SkyReels-V2: Infinite-length Film Generative Model
by: Chen, Guibin, et al.
Published: (2025)

Video Diffusion Transformers are In-Context Learners
by: Fei, Zhengcong, et al.
Published: (2024)

Ingredients: Blending Custom Photos with Video Diffusion Transformers
by: Fei, Zhengcong, et al.
Published: (2025)

Dimba: Transformer-Mamba Diffusion Models
by: Fei, Zhengcong, et al.
Published: (2024)

SkyReels-Text: Fine-Grained Font-Controllable Text Editing for Poster Design
by: Yu, Yunjie, et al.
Published: (2025)

Scaling Diffusion Transformers to 16 Billion Parameters
by: Fei, Zhengcong, et al.
Published: (2024)

Diffusion-RWKV: Scaling RWKV-Like Architectures for Diffusion Models
by: Fei, Zhengcong, et al.
Published: (2024)

UniTalking: A Unified Audio-Video Framework for Talking Portrait Generation
by: Li, Hebeizi, et al.
Published: (2026)

AudioGen-Omni: A Unified Multimodal Diffusion Transformer for Video-Synchronized Audio, Speech, and Song Generation
by: Wang, Le, et al.
Published: (2025)

GMTalker: Gaussian Mixture-based Audio-Driven Emotional Talking Video Portraits
by: Xia, Yibo, et al.
Published: (2023)

RAP: Real-time Audio-driven Portrait Animation with Video Diffusion Transformer
by: Du, Fangyu, et al.
Published: (2025)

EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditions
by: Tian, Linrui, et al.
Published: (2024)

PortraitTalk: Towards Customizable One-Shot Audio-to-Talking Face Generation
by: Nazarieh, Fatemeh, et al.
Published: (2024)

Scalable Diffusion Models with State Space Backbone
by: Fei, Zhengcong, et al.
Published: (2024)

OmniNFT: Modality-wise Omni Diffusion Reinforcement for Joint Audio-Video Generation
by: Zhang, Guohui, et al.
Published: (2026)

EditYourself: Audio-Driven Generation and Manipulation of Talking Head Videos with Diffusion Transformers
by: Flynn, John, et al.
Published: (2026)

A-JEPA: Joint-Embedding Predictive Architecture Can Listen
by: Fei, Zhengcong, et al.
Published: (2023)

Music Consistency Models
by: Fei, Zhengcong, et al.
Published: (2024)

FlowPortrait: Reinforcement Learning for Audio-Driven Portrait Video Generation
by: Tan, Weiting, et al.
Published: (2026)

Audio-Driven Talking Face Video Generation with Joint Uncertainty Learning
by: Xie, Yifan, et al.
Published: (2025)

OmniAudio: Generating Spatial Audio from 360-Degree Video
by: Liu, Huadai, et al.
Published: (2025)

Comparative Analysis of Audio Feature Extraction for Real-Time Talking Portrait Synthesis
by: Salehi, Pegah, et al.
Published: (2024)

FLOAT: Generative Motion Latent Flow Matching for Audio-driven Talking Portrait
by: Ki, Taekyung, et al.
Published: (2024)

Unlock Pose Diversity: Accurate and Efficient Implicit Keypoint-based Spatiotemporal Diffusion for Audio-driven Talking Portrait
by: Yang, Chaolong, et al.
Published: (2025)

OmniCustom: Sync Audio-Video Customization Via Joint Audio-Video Generation Model
by: Li, Maomao, et al.
Published: (2026)

Omni-Judge: Can Omni-LLMs Serve as Human-Aligned Judges for Text-Conditioned Audio-Video Generation?
by: Liang, Susan, et al.
Published: (2026)

Audio-visual Event Localization on Portrait Mode Short Videos
by: Liu, Wuyang, et al.
Published: (2025)

EmotiveTalk: Expressive Talking Head Generation through Audio Information Decoupling and Emotional Video Diffusion
by: Wang, Haotian, et al.
Published: (2024)

OmniVideoBench: Towards Audio-Visual Understanding Evaluation for Omni MLLMs
by: Li, Caorui, et al.
Published: (2025)

In-Context Audio Control of Video Diffusion Transformers
by: Liu, Wenze, et al.
Published: (2025)

FantasyTalking2: Timestep-Layer Adaptive Preference Optimization for Audio-Driven Portrait Animation
by: Wang, MengChao, et al.
Published: (2025)

FREAK: Frequency-modulated High-fidelity and Real-time Audio-driven Talking Portrait Synthesis
by: Ni, Ziqi, et al.
Published: (2025)

OmniTalker: One-shot Real-time Text-Driven Talking Audio-Video Generation With Multimodal Style Mimicking
by: Wang, Zhongjian, et al.
Published: (2025)

FLUX that Plays Music
by: Fei, Zhengcong, et al.
Published: (2024)

IM-Portrait: Learning 3D-aware Video Diffusion for Photorealistic Talking Heads from Monocular Videos
by: Li, Yuan, et al.
Published: (2025)

AV-DiT: Efficient Audio-Visual Diffusion Transformer for Joint Audio and Video Generation
by: Wang, Kai, et al.
Published: (2024)