Copertina

UniSonate: A Unified Model for Speech, Music, and Sound Effect Generation with Text Instructions

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Qiang, Chunyu, Wang, Xiaopeng, Yin, Kang, Liang, Yuzhe, Guo, Yuxin, Ma, Teng, Zhang, Ziyu, Wang, Tianrui, Gong, Cheng, Chen, Yushen, Fu, Ruibo, Zhang, Chen, Wang, Longbiao, Dang, Jianwu
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Audio and Speech Processing Artificial Intelligence Computation and Language Sound
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

InstructAudio: Unified speech and music generation with natural language instruction
di: Qiang, Chunyu, et al.
Pubblicazione: (2025)

MM-Sonate: Multimodal Controllable Audio-Video Generation with Zero-Shot Voice Cloning
di: Qiang, Chunyu, et al.
Pubblicazione: (2026)

SecoustiCodec: Cross-Modal Aligned Streaming Single-Codecbook Speech Codec
di: Qiang, Chunyu, et al.
Pubblicazione: (2025)

Expressive Prompting: Improving Emotion Intensity and Speaker Consistency in Zero-Shot TTS
di: Wang, Haoyu, et al.
Pubblicazione: (2024)

Mamba-SEUNet: Mamba UNet for Monaural Speech Enhancement
di: Wang, Junyu, et al.
Pubblicazione: (2024)

LORT: Locally Refined Convolution and Taylor Transformer for Monaural Speech Enhancement
di: Wang, Junyu, et al.
Pubblicazione: (2025)

VQ-CTAP: Cross-Modal Fine-Grained Sequence Representation Learning for Speech Processing
di: Qiang, Chunyu, et al.
Pubblicazione: (2024)

ASDA: Audio Spectrogram Differential Attention Mechanism for Self-Supervised Representation Learning
di: Wang, Junyu, et al.
Pubblicazione: (2025)

Perturbation Self-Supervised Representations for Cross-Lingual Emotion TTS: Stage-Wise Modeling of Emotion and Speaker
di: Gong, Cheng, et al.
Pubblicazione: (2025)

Word-Level Emotional Expression Control in Zero-Shot Text-to-Speech Synthesis
di: Wang, Tianrui, et al.
Pubblicazione: (2025)

Reducing the Gap Between Pretrained Speech Enhancement and Recognition Models Using a Real Speech-Trained Bridging Module
di: Cui, Zhongjian, et al.
Pubblicazione: (2025)

Habibi: Laying the Open-Source Foundation of Unified-Dialectal Arabic Speech Synthesis
di: Chen, Yushen, et al.
Pubblicazione: (2026)

MSR-HuBERT: Self-supervised Pre-training for Adaptation to Multiple Sampling Rates
di: Huang, Zikang, et al.
Pubblicazione: (2026)

Progressive Residual Extraction based Pre-training for Speech Representation Learning
di: Wang, Tianrui, et al.
Pubblicazione: (2024)

ZMM-TTS: Zero-shot Multilingual and Multispeaker Speech Synthesis Conditioned on Self-supervised Discrete Speech Representations
di: Gong, Cheng, et al.
Pubblicazione: (2023)

Error Correction by Paying Attention to Both Acoustic and Confidence References for Automatic Speech Recognition
di: Shu, Yuchun, et al.
Pubblicazione: (2024)

Efficient Emotion and Speaker Adaptation in LLM-Based TTS via Characteristic-Specific Partial Fine-Tuning
di: Wang, Tianrui, et al.
Pubblicazione: (2025)

Enriching Multimodal Sentiment Analysis through Textual Emotional Descriptions of Visual-Audio Content
di: Wu, Sheng, et al.
Pubblicazione: (2024)

POTSA: A Cross-Lingual Speech Alignment Framework for Speech-to-Text Translation
di: Li, Xuanchen, et al.
Pubblicazione: (2025)

UniMoE-Audio: Unified Speech and Music Generation with Dynamic-Capacity MoE
di: Liu, Zhenyu, et al.
Pubblicazione: (2025)

DMP-TTS: Disentangled multi-modal Prompting for Controllable Text-to-Speech with Chained Guidance
di: Yin, Kang, et al.
Pubblicazione: (2025)

Breaking Data Efficiency Dilemma: A Federated and Augmented Learning Framework For Alzheimer's Disease Detection via Speech
di: Wei, Xiao, et al.
Pubblicazione: (2026)

AIMDiT: Modality Augmentation and Interaction via Multimodal Dimension Transformation for Emotion Recognition in Conversations
di: Wu, Sheng, et al.
Pubblicazione: (2024)

Separate First, Fuse Later: Mitigating Cross-Modal Interference in Audio-Visual LLMs Reasoning with Modality-Specific Chain-of-Thought
di: Li, Xuanchen, et al.
Pubblicazione: (2026)

An Initial Investigation of Language Adaptation for TTS Systems under Low-resource Scenarios
di: Gong, Cheng, et al.
Pubblicazione: (2024)

CECOR: Correction-oriented synthetic data construction for factual error correction
di: Zhu, Lei, et al.
Pubblicazione: (2026)

M3-TTS: Multi-modal DiT Alignment & Mel-latent for Zero-shot High-fidelity Speech Synthesis
di: Wang, Xiaopeng, et al.
Pubblicazione: (2025)

AudioGen-Omni: A Unified Multimodal Diffusion Transformer for Video-Synchronized Audio, Speech, and Song Generation
di: Wang, Le, et al.
Pubblicazione: (2025)

Rethinking Contrastive Learning in Graph Anomaly Detection: A Clean-View Perspective
di: Jin, Di, et al.
Pubblicazione: (2025)

UniSTPA: A Safety Analysis Framework for End-to-End Autonomous Driving
di: Kou, Hongrui, et al.
Pubblicazione: (2025)

Towards Flow-Matching-based TTS without Classifier-Free Guidance
di: Liang, Yuzhe, et al.
Pubblicazione: (2025)

UniSS: Unified Expressive Speech-to-Speech Translation with Your Voice
di: Cheng, Sitong, et al.
Pubblicazione: (2025)

UniMuMo: Unified Text, Music and Motion Generation
di: Yang, Han, et al.
Pubblicazione: (2024)

Uni-NTFM: A Unified Foundation Model for EEG Signal Representation Learning
di: Chen, Zhisheng, et al.
Pubblicazione: (2025)

Integration of Old and New Knowledge for Generalized Intent Discovery: A Consistency-driven Prototype-Prompting Framework
di: Wei, Xiao, et al.
Pubblicazione: (2025)

Score-Informed Musical Source Separation and Reconstruction
di: Han, Yushen
Pubblicazione: (2013)

UniVocal: Unified Speech-Singing Code-Switching Synthesis
di: Shi, Yufei, et al.
Pubblicazione: (2026)

Evaluating the Expressive Appropriateness of Speech in Rich Contexts
di: Wang, Tianrui, et al.
Pubblicazione: (2026)

Apollo: Unified Multi-Task Audio-Video Joint Generation
di: Wang, Jun, et al.
Pubblicazione: (2026)

Talking Head Generation Driven by Speech-Related Facial Action Units and Audio- Based on Multimodal Representation Fusion
di: Chen, Sen, et al.
Pubblicazione: (2022)