Imagen de Portada

A Versatile Diffusion Transformer with Mixture of Noise Levels for Audiovisual Generation

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Kim, Gwanghyun, Martinez, Alonso, Su, Yu-Chuan, Jou, Brendan, Lezama, José, Gupta, Agrim, Yu, Lijun, Jiang, Lu, Jansen, Aren, Walker, Jacob, Somandepalli, Krishna
Formato:	Preprint
Publicado:	2024
Materias:	Computer Vision and Pattern Recognition Machine Learning Multimedia Sound Audio and Speech Processing
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Noise-Conditioned Mixture-of-Experts Framework for Robust Speaker Verification
por: Gu, Bin, et al.
Publicado: (2025)

Face-Voice Association for Audiovisual Active Speaker Detection in Egocentric Recordings
por: Clarke, Jason, et al.
Publicado: (2025)

Cross-Modal Watermarking for Authentic Audio Recovery and Tamper Localization in Synthesized Audiovisual Forgeries
por: Kim, Minyoung, et al.
Publicado: (2025)

VERSA: A Versatile Evaluation Toolkit for Speech, Audio, and Music
por: Shi, Jiatong, et al.
Publicado: (2024)

Efficient Feature Extraction and Late Fusion Strategy for Audiovisual Emotional Mimicry Intensity Estimation
por: Yu, Jun, et al.
Publicado: (2024)

Multi-Track MusicLDM: Towards Versatile Music Generation with Latent Diffusion Model
por: Karchkhadze, Tornike, et al.
Publicado: (2024)

MEGADance: Mixture-of-Experts Architecture for Genre-Aware 3D Dance Generation
por: Yang, Kaixing, et al.
Publicado: (2025)

STA-V2A: Video-to-Audio Generation with Semantic and Temporal Alignment
por: Ren, Yong, et al.
Publicado: (2024)

Robust Wake Word Spotting With Frame-Level Cross-Modal Attention Based Audio-Visual Conformer
por: Wang, Haoxu, et al.
Publicado: (2024)

ScripTONES: Sentiment-Conditioned Music Generation for Movie Scripts
por: Veerendranath, Vishruth, et al.
Publicado: (2024)

LCB-net: Long-Context Biasing for Audio-Visual Speech Recognition
por: Yu, Fan, et al.
Publicado: (2024)

SonicVisionLM: Playing Sound with Vision Language Models
por: Xie, Zhifeng, et al.
Publicado: (2024)

Mitigating Audiovisual Mismatch in Visual-Guide Audio Captioning
por: Xu, Le, et al.
Publicado: (2025)

Singer separation for karaoke content generation
por: Lin, Hsuan-Yu, et al.
Publicado: (2021)

RVCBench: Benchmarking the Robustness of Voice Cloning Across Modern Audio Generation Models
por: Jin, Ruinan, et al.
Publicado: (2026)

Dopamine Audiobook: A Training-free MLLM Agent for Emotional and Immersive Audiobook Generation
por: Rong, Yan, et al.
Publicado: (2025)

Learning What To Hear: Boosting Sound-Source Association For Robust Audiovisual Instance Segmentation
por: Seo, Jinbae, et al.
Publicado: (2025)

Speech-to-See: End-to-End Speech-Driven Open-Set Object Detection
por: Lu, Wenhuan, et al.
Publicado: (2025)

ISDrama: Immersive Spatial Drama Generation through Multimodal Prompting
por: Zhang, Yu, et al.
Publicado: (2025)

FGAS: Fixed Decoder Network-Based Audio Steganography with Adversarial Perturbation Generation
por: Yan, Jialin, et al.
Publicado: (2025)

Towards Streaming Synchronized Spatial Audio Generation via Autoregressive Diffusion Transformer
por: Lei, Ke, et al.
Publicado: (2026)

Attentive-based Multi-level Feature Fusion for Voice Disorder Diagnosis
por: Shen, Lipeng, et al.
Publicado: (2024)

Versatile audio-visual learning for emotion recognition
por: Goncalves, Lucas, et al.
Publicado: (2023)

A Study on Synthesizing Expressive Violin Performances: Approaches and Comparisons
por: Hung, Tzu-Yun, et al.
Publicado: (2024)

MusicAOG: an Energy-Based Model for Learning and Sampling a Hierarchical Representation of Symbolic Music
por: Qian, Yikai, et al.
Publicado: (2024)

A Unified Framework for Modality-Agnostic Deepfakes Detection
por: Yu, Cai, et al.
Publicado: (2023)

Hearing from Silence: Reasoning Audio Descriptions from Silent Videos via Vision-Language Model
por: Ren, Yong, et al.
Publicado: (2025)

Efficient Audiovisual Speech Processing via MUTUD: Multimodal Training and Unimodal Deployment
por: Hong, Joanna, et al.
Publicado: (2025)

Double Mixture: Towards Continual Event Detection from Speech
por: Kang, Jingqi, et al.
Publicado: (2024)

Audio-Visual Speaker Tracking: Progress, Challenges, and Future Directions
por: Zhao, Jinzheng, et al.
Publicado: (2023)

Video-Guided Text-to-Music Generation Using Public Domain Movie Collections
por: Kim, Haven, et al.
Publicado: (2025)

PerformSinger: Multimodal Singing Voice Synthesis Leveraging Synchronized Lip Cues from Singing Performance Videos
por: Gu, Ke, et al.
Publicado: (2025)

Efficient Speech Watermarking for Speech Synthesis via Progressive Knowledge Distillation
por: Cui, Yang, et al.
Publicado: (2025)

ARECHO: Autoregressive Evaluation via Chain-Based Hypothesis Optimization for Speech Multi-Metric Estimation
por: Shi, Jiatong, et al.
Publicado: (2025)

SteerMusic: Enhanced Musical Consistency for Zero-shot Text-guided and Personalized Music Editing
por: Niu, Xinlei, et al.
Publicado: (2025)

Building Audio-Visual Digital Twins with Smartphones
por: Lan, Zitong, et al.
Publicado: (2025)

MiniMind-O Technical Report: An Open Small-Scale Speech-Native Omni Model
por: Gong, Jingyao
Publicado: (2026)

Dance2MIDI: Dance-driven multi-instruments music generation
por: Han, Bo, et al.
Publicado: (2023)

Listening Between the Lines: Synthetic Speech Detection Disregarding Verbal Content
por: Salvi, Davide, et al.
Publicado: (2024)

LRS-VoxMM: A benchmark for in-the-wild audio-visual speech recognition
por: Kwak, Doyeop, et al.
Publicado: (2026)