Cover Image

Diffusion Models as Masked Audio-Video Learners

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Nunez, Elvis, Jin, Yanzi, Rastegari, Mohammad, Mehta, Sachin, Horton, Maxwell
Format:	Preprint
Published:	2023
Subjects:	Sound Computer Vision and Pattern Recognition Multimedia Audio and Speech Processing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Large Language Models are Strong Audio-Visual Speech Recognition Learners
by: Cappellazzo, Umberto, et al.
Published: (2024)

AudioGen-Omni: A Unified Multimodal Diffusion Transformer for Video-Synchronized Audio, Speech, and Song Generation
by: Wang, Le, et al.
Published: (2025)

Draw an Audio: Leveraging Multi-Instruction for Video-to-Audio Synthesis
by: Yang, Qi, et al.
Published: (2024)

Video-to-Audio Generation with Hidden Alignment
by: Xu, Manjie, et al.
Published: (2024)

Temporally Aligned Audio for Video with Autoregression
by: Viertola, Ilpo, et al.
Published: (2024)

Efficient Video-to-Audio Generation via Multiple Foundation Models Mapper
by: Chen, Gehui, et al.
Published: (2025)

CAV-MAE Sync: Improving Contrastive Audio-Visual Mask Autoencoders via Fine-Grained Alignment
by: Araujo, Edson, et al.
Published: (2025)

Contrastive Conditional Latent Diffusion for Audio-visual Segmentation
by: Mao, Yuxin, et al.
Published: (2023)

EgoSonics: Generating Synchronized Audio for Silent Egocentric Videos
by: Rai, Aashish, et al.
Published: (2024)

AVFF: Audio-Visual Feature Fusion for Video Deepfake Detection
by: Oorloff, Trevine, et al.
Published: (2024)

Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video Generation
by: Low, Chetwin, et al.
Published: (2025)

Audio-Assisted Face Video Restoration with Temporal and Identity Complementary Learning
by: Cao, Yuqin, et al.
Published: (2025)

Frieren: Efficient Video-to-Audio Generation Network with Rectified Flow Matching
by: Wang, Yongqi, et al.
Published: (2024)

VinTAGe: Joint Video and Text Conditioning for Holistic Audio Generation
by: Kushwaha, Saksham Singh, et al.
Published: (2024)

Audio-Visual Speech Enhancement In Complex Scenarios With Separation And Dereverberation Joint Modeling
by: Du, Jiarong, et al.
Published: (2025)

VAInpaint: Zero-Shot Video-Audio inpainting framework with LLMs-driven Module
by: Wu, Kam Man, et al.
Published: (2025)

Seeing and Hearing: Open-domain Visual-Audio Generation with Diffusion Latent Aligners
by: Xing, Yazhou, et al.
Published: (2024)

UniAV: Unified Audio-Visual Perception for Multi-Task Video Event Localization
by: Geng, Tiantian, et al.
Published: (2024)

SlideAVSR: A Dataset of Paper Explanation Videos for Audio-Visual Speech Recognition
by: Wang, Hao, et al.
Published: (2024)

CMMD: Contrastive Multi-Modal Diffusion for Video-Audio Conditional Modeling
by: Yang, Ruihan, et al.
Published: (2023)

Tri-Ergon: Fine-grained Video-to-Audio Generation with Multi-modal Conditions and LUFS Control
by: Li, Bingliang, et al.
Published: (2024)

On the Audio Hallucinations in Large Audio-Video Language Models
by: Nishimura, Taichi, et al.
Published: (2024)

Nexus: An Omni-Perceptive And -Interactive Model for Language, Audio, And Vision
by: Liu, Che, et al.
Published: (2025)

Talker-T2AV: Joint Talking Audio-Video Generation with Autoregressive Diffusion Modeling
by: Ye, Zhen, et al.
Published: (2026)

Aligned Better, Listen Better for Audio-Visual Large Language Models
by: Guo, Yuxin, et al.
Published: (2025)

MMS-LLaMA: Efficient LLM-based Audio-Visual Speech Recognition with Minimal Multimodal Speech Tokens
by: Yeo, Jeong Hun, et al.
Published: (2025)

Weakly-supervised Audio Temporal Forgery Localization via Progressive Audio-language Co-learning Network
by: Wu, Junyan, et al.
Published: (2025)

AV-SUPERB: A Multi-Task Evaluation Benchmark for Audio-Visual Representation Models
by: Tseng, Yuan, et al.
Published: (2023)

3D Audio-Visual Segmentation
by: Sokolov, Artem, et al.
Published: (2024)

AGAV-Rater: Adapting Large Multimodal Model for AI-Generated Audio-Visual Quality Assessment
by: Cao, Yuqin, et al.
Published: (2025)

Statistics-aware Audio-visual Deepfake Detector
by: Astrid, Marcella, et al.
Published: (2024)

Benchmarking Cross-Domain Audio-Visual Deception Detection
by: Guo, Xiaobao, et al.
Published: (2024)

Audio-Vision Contrastive Learning for Phonological Class Recognition
by: Liu, Daiqi, et al.
Published: (2025)

Mitigating Audiovisual Mismatch in Visual-Guide Audio Captioning
by: Xu, Le, et al.
Published: (2025)

Detecting Audio-Visual Deepfakes with Fine-Grained Inconsistencies
by: Astrid, Marcella, et al.
Published: (2024)

Learning Self-Supervised Audio-Visual Representations for Sound Recommendations
by: Krishnamurthy, Sudha
Published: (2024)

DETECLAP: Enhancing Audio-Visual Representation Learning with Object Information
by: Nakada, Shota, et al.
Published: (2024)

UniForm: A Unified Multi-Task Diffusion Transformer for Audio-Video Generation
by: Zhao, Lei, et al.
Published: (2025)

Spiking Tucker Fusion Transformer for Audio-Visual Zero-Shot Learning
by: Li, Wenrui, et al.
Published: (2024)

Pilot-guided Multimodal Semantic Communication for Audio-Visual Event Localization
by: Yu, Fei, et al.
Published: (2024)