Image de couverture de livre

SpeechCaps: Advancing Instruction-Based Universal Speech Models with Multi-Talker Speaking Style Captioning

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Huang, Chien-yu, Shih, Min-Han, Lu, Ke-Han, Hsiao, Chi-Yuan, Lee, Hung-yi
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language Audio and Speech Processing
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Speech-IFEval: Evaluating Instruction-Following and Quantifying Catastrophic Forgetting in Speech-Aware Language Models
par: Lu, Ke-Han, et autres
Publié: (2025)

Fusion of Discrete Representations and Self-Augmented Representations for Multilingual Automatic Speech Recognition
par: Wang, Shih-heng, et autres
Publié: (2024)

Dynamic-SUPERB: Towards A Dynamic, Collaborative, and Comprehensive Instruction-Tuning Benchmark for Speech
par: Huang, Chien-yu, et autres
Publié: (2023)

Advancing Large Language Models to Capture Varied Speaking Styles and Respond Properly in Spoken Conversations
par: Lin, Guan-Ting, et autres
Publié: (2024)

CapSpeech: Enabling Downstream Applications in Style-Captioned Text-to-Speech
par: Wang, Helin, et autres
Publié: (2025)

ASPIRin: Action Space Projection for Interactivity-Optimized Reinforcement Learning in Full-Duplex Speech Language Models
par: Hsiao, Chi-Yuan, et autres
Publié: (2026)

Investigating Zero-Shot Generalizability on Mandarin-English Code-Switched ASR and Speech-to-text Translation of Recent Foundation Models with Self-Supervision and Weak Supervision
par: Yang, Chih-Kai, et autres
Publié: (2023)

DeSTA2: Developing Instruction-Following Speech Language Model Without Speech Instruction-Tuning Data
par: Lu, Ke-Han, et autres
Publié: (2024)

Gender Bias in Instruction-Guided Speech Synthesis Models
par: Kuan, Chun-Yi, et autres
Publié: (2025)

Speech-Copilot: Leveraging Large Language Models for Speech Processing via Task Decomposition, Modularization, and Program Generation
par: Kuan, Chun-Yi, et autres
Publié: (2024)

DeSTA: Enhancing Speech Language Models through Descriptive Speech-Text Alignment
par: Lu, Ke-Han, et autres
Publié: (2024)

Continual Test-time Adaptation for End-to-end Speech Recognition on Noisy Speech
par: Lin, Guan-Ting, et autres
Publié: (2024)

UniTalker: Conversational Speech-Visual Synthesis
par: Hu, Yifan, et autres
Publié: (2025)

CodecFake: Enhancing Anti-Spoofing Models Against Deepfake Audios from Codec-Based Speech Synthesis Systems
par: Wu, Haibin, et autres
Publié: (2024)

Toward Fair Speech Technologies: A Comprehensive Survey of Bias and Fairness in Speech AI
par: Lin, Yi-Cheng, et autres
Publié: (2026)

Listen and Speak Fairly: A Study on Semantic Gender Bias in Speech Integrated Large Language Models
par: Lin, Yi-Cheng, et autres
Publié: (2024)

SMILE: Speech Meta In-Context Learning for Low-Resource Language Automatic Speech Recognition
par: Hsu, Ming-Hao, et autres
Publié: (2024)

Do You Hear What I Mean? Quantifying the Instruction-Perception Gap in Instruction-Guided Expressive Text-To-Speech Systems
par: Lin, Yi-Cheng, et autres
Publié: (2025)

Emo-bias: A Large Scale Evaluation of Social Bias on Speech Emotion Recognition
par: Lin, Yi-Cheng, et autres
Publié: (2024)

Universal Robust Speech Adaptation for Cross-Domain Speech Recognition and Enhancement
par: Wang, Chien-Chun, et autres
Publié: (2026)

Multi-Distillation from Speech and Music Representation Models
par: Wei, Jui-Chiang, et autres
Publié: (2025)

Parallel Synthesis for Autoregressive Speech Generation
par: Hsu, Po-chun, et autres
Publié: (2022)

SPAR-K: Scheduled Periodic Alternating Early Exit for Spoken Language Models
par: Huang, Hsiao-Ying, et autres
Publié: (2026)

AlignCap: Aligning Speech Emotion Captioning to Human Preferences
par: Liang, Ziqi, et autres
Publié: (2024)

ISSE: An Instruction-Guided Speech Style Editing Dataset And Benchmark
par: Chen, Yun, et autres
Publié: (2025)

AS-Speech: Adaptive Style For Speech Synthesis
par: Li, Zhipeng, et autres
Publié: (2024)

Full-Duplex-Bench v1.5: Evaluating Overlap Handling for Full-Duplex Speech Models
par: Lin, Guan-Ting, et autres
Publié: (2025)

LibriTTS-P: A Corpus with Speaking Style and Speaker Identity Prompts for Text-to-Speech and Style Captioning
par: Kawamura, Masaya, et autres
Publié: (2024)

Factor-Conditioned Speaking-Style Captioning
par: Ando, Atsushi, et autres
Publié: (2024)

JIS: A Speech Corpus of Japanese Idol Speakers with Various Speaking Styles
par: Kondo, Yuto, et autres
Publié: (2025)

End-to-End DOA-Guided Speech Extraction in Noisy Multi-Talker Scenarios
par: Jing, Kangqi, et autres
Publié: (2025)

Improving Speech Emotion Recognition in Under-Resourced Languages via Speech-to-Speech Translation with Bootstrapping Data Selection
par: Lin, Hsi-Che, et autres
Publié: (2024)

ParaS2S: Benchmarking and Aligning Spoken Language Models for Paralinguistic-aware Speech-to-Speech Interaction
par: Yang, Shu-wen, et autres
Publié: (2025)

Thinking in Directivity: Speech Large Language Model for Multi-Talker Directional Speech Recognition
par: Xie, Jiamin, et autres
Publié: (2025)

Chain-Talker: Chain Understanding and Rendering for Empathetic Conversational Speech Synthesis
par: Hu, Yifan, et autres
Publié: (2025)

Reducing Object Hallucination in Large Audio-Language Models via Audio-Aware Decoding
par: Hsu, Tzu-wen, et autres
Publié: (2025)

Reasoning Beyond Majority Vote: An Explainable SpeechLM Framework for Speech Emotion Recognition
par: Su, Bo-Hao, et autres
Publié: (2025)

Empowering Whisper as a Joint Multi-Talker and Target-Talker Speech Recognition System
par: Meng, Lingwei, et autres
Publié: (2024)

Spoken Stereoset: On Evaluating Social Bias Toward Speaker in Speech Large Language Models
par: Lin, Yi-Cheng, et autres
Publié: (2024)

ASR for Affective Speech: Investigating Impact of Emotion and Speech Generative Strategy
par: Wu, Ya-Tse, et autres
Publié: (2026)