Imagen de Portada

SpeechCaps: Advancing Instruction-Based Universal Speech Models with Multi-Talker Speaking Style Captioning

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Huang, Chien-yu, Shih, Min-Han, Lu, Ke-Han, Hsiao, Chi-Yuan, Lee, Hung-yi
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language Audio and Speech Processing
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Speech-IFEval: Evaluating Instruction-Following and Quantifying Catastrophic Forgetting in Speech-Aware Language Models
por: Lu, Ke-Han, et al.
Publicado: (2025)

Fusion of Discrete Representations and Self-Augmented Representations for Multilingual Automatic Speech Recognition
por: Wang, Shih-heng, et al.
Publicado: (2024)

Dynamic-SUPERB: Towards A Dynamic, Collaborative, and Comprehensive Instruction-Tuning Benchmark for Speech
por: Huang, Chien-yu, et al.
Publicado: (2023)

Advancing Large Language Models to Capture Varied Speaking Styles and Respond Properly in Spoken Conversations
por: Lin, Guan-Ting, et al.
Publicado: (2024)

CapSpeech: Enabling Downstream Applications in Style-Captioned Text-to-Speech
por: Wang, Helin, et al.
Publicado: (2025)

ASPIRin: Action Space Projection for Interactivity-Optimized Reinforcement Learning in Full-Duplex Speech Language Models
por: Hsiao, Chi-Yuan, et al.
Publicado: (2026)

Investigating Zero-Shot Generalizability on Mandarin-English Code-Switched ASR and Speech-to-text Translation of Recent Foundation Models with Self-Supervision and Weak Supervision
por: Yang, Chih-Kai, et al.
Publicado: (2023)

DeSTA2: Developing Instruction-Following Speech Language Model Without Speech Instruction-Tuning Data
por: Lu, Ke-Han, et al.
Publicado: (2024)

Gender Bias in Instruction-Guided Speech Synthesis Models
por: Kuan, Chun-Yi, et al.
Publicado: (2025)

Speech-Copilot: Leveraging Large Language Models for Speech Processing via Task Decomposition, Modularization, and Program Generation
por: Kuan, Chun-Yi, et al.
Publicado: (2024)

DeSTA: Enhancing Speech Language Models through Descriptive Speech-Text Alignment
por: Lu, Ke-Han, et al.
Publicado: (2024)

Continual Test-time Adaptation for End-to-end Speech Recognition on Noisy Speech
por: Lin, Guan-Ting, et al.
Publicado: (2024)

UniTalker: Conversational Speech-Visual Synthesis
por: Hu, Yifan, et al.
Publicado: (2025)

CodecFake: Enhancing Anti-Spoofing Models Against Deepfake Audios from Codec-Based Speech Synthesis Systems
por: Wu, Haibin, et al.
Publicado: (2024)

Toward Fair Speech Technologies: A Comprehensive Survey of Bias and Fairness in Speech AI
por: Lin, Yi-Cheng, et al.
Publicado: (2026)

Listen and Speak Fairly: A Study on Semantic Gender Bias in Speech Integrated Large Language Models
por: Lin, Yi-Cheng, et al.
Publicado: (2024)

SMILE: Speech Meta In-Context Learning for Low-Resource Language Automatic Speech Recognition
por: Hsu, Ming-Hao, et al.
Publicado: (2024)

Do You Hear What I Mean? Quantifying the Instruction-Perception Gap in Instruction-Guided Expressive Text-To-Speech Systems
por: Lin, Yi-Cheng, et al.
Publicado: (2025)

Emo-bias: A Large Scale Evaluation of Social Bias on Speech Emotion Recognition
por: Lin, Yi-Cheng, et al.
Publicado: (2024)

Universal Robust Speech Adaptation for Cross-Domain Speech Recognition and Enhancement
por: Wang, Chien-Chun, et al.
Publicado: (2026)

Multi-Distillation from Speech and Music Representation Models
por: Wei, Jui-Chiang, et al.
Publicado: (2025)

Parallel Synthesis for Autoregressive Speech Generation
por: Hsu, Po-chun, et al.
Publicado: (2022)

AlignCap: Aligning Speech Emotion Captioning to Human Preferences
por: Liang, Ziqi, et al.
Publicado: (2024)

SPAR-K: Scheduled Periodic Alternating Early Exit for Spoken Language Models
por: Huang, Hsiao-Ying, et al.
Publicado: (2026)

ISSE: An Instruction-Guided Speech Style Editing Dataset And Benchmark
por: Chen, Yun, et al.
Publicado: (2025)

AS-Speech: Adaptive Style For Speech Synthesis
por: Li, Zhipeng, et al.
Publicado: (2024)

Full-Duplex-Bench v1.5: Evaluating Overlap Handling for Full-Duplex Speech Models
por: Lin, Guan-Ting, et al.
Publicado: (2025)

LibriTTS-P: A Corpus with Speaking Style and Speaker Identity Prompts for Text-to-Speech and Style Captioning
por: Kawamura, Masaya, et al.
Publicado: (2024)

Factor-Conditioned Speaking-Style Captioning
por: Ando, Atsushi, et al.
Publicado: (2024)

JIS: A Speech Corpus of Japanese Idol Speakers with Various Speaking Styles
por: Kondo, Yuto, et al.
Publicado: (2025)

End-to-End DOA-Guided Speech Extraction in Noisy Multi-Talker Scenarios
por: Jing, Kangqi, et al.
Publicado: (2025)

Improving Speech Emotion Recognition in Under-Resourced Languages via Speech-to-Speech Translation with Bootstrapping Data Selection
por: Lin, Hsi-Che, et al.
Publicado: (2024)

ParaS2S: Benchmarking and Aligning Spoken Language Models for Paralinguistic-aware Speech-to-Speech Interaction
por: Yang, Shu-wen, et al.
Publicado: (2025)

Thinking in Directivity: Speech Large Language Model for Multi-Talker Directional Speech Recognition
por: Xie, Jiamin, et al.
Publicado: (2025)

Chain-Talker: Chain Understanding and Rendering for Empathetic Conversational Speech Synthesis
por: Hu, Yifan, et al.
Publicado: (2025)

Reducing Object Hallucination in Large Audio-Language Models via Audio-Aware Decoding
por: Hsu, Tzu-wen, et al.
Publicado: (2025)

Reasoning Beyond Majority Vote: An Explainable SpeechLM Framework for Speech Emotion Recognition
por: Su, Bo-Hao, et al.
Publicado: (2025)

Empowering Whisper as a Joint Multi-Talker and Target-Talker Speech Recognition System
por: Meng, Lingwei, et al.
Publicado: (2024)

Spoken Stereoset: On Evaluating Social Bias Toward Speaker in Speech Large Language Models
por: Lin, Yi-Cheng, et al.
Publicado: (2024)

ASR for Affective Speech: Investigating Impact of Emotion and Speech Generative Strategy
por: Wu, Ya-Tse, et al.
Publicado: (2026)