Cover Image

VocalBench-DF: A Benchmark for Evaluating Speech LLM Robustness to Disfluency

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Hongcheng, Hou, Yixuan, Liu, Heyang, Wang, Yuhao, Wang, Yanfeng, Wang, Yu
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

VocalBench: Benchmarking the Vocal Conversational Abilities for Speech Interaction Models
by: Liu, Heyang, et al.
Published: (2025)

VocalBench-zh: Decomposing and Benchmarking the Speech Conversational Abilities in Mandarin Context
by: Liu, Heyang, et al.
Published: (2025)

VocalNet-MDM: Accelerating Streaming Speech LLM via Self-Distilled Masked Diffusion Modeling
by: Cheng, Ziyang, et al.
Published: (2026)

CS3-Bench: Evaluating and Enhancing Speech-to-Speech LLMs for Mandarin-English Code-Switching
by: Liu, Heyang, et al.
Published: (2025)

MM-SAP: A Comprehensive Benchmark for Assessing Self-Awareness of Multimodal Large Language Models in Perception
by: Wang, Yuhao, et al.
Published: (2024)

VocalNet: Speech LLM with Multi-Token Prediction for Faster and High-Quality Generation
by: Wang, Yuhao, et al.
Published: (2025)

Med-PMC: Medical Personalized Multi-modal Consultation with a Proactive Ask-First-Observe-Next Paradigm
by: Liu, Hongcheng, et al.
Published: (2024)

Decoding Linguistic Representations of Human Brain
by: Wang, Yu, et al.
Published: (2024)

Drawing the Line: Enhancing Trustworthiness of MLLMs Through the Power of Refusal
by: Wang, Yuhao, et al.
Published: (2024)

Post-decoder Biasing for End-to-End Speech Recognition of Multi-turn Medical Interview
by: Liu, Heyang, et al.
Published: (2024)

SOVA-Bench: Benchmarking the Speech Conversation Ability for LLM-based Voice Assistant
by: Hou, Yixuan, et al.
Published: (2025)

VocalNet-M2: Advancing Low-Latency Spoken Language Modeling via Integrated Multi-Codebook Tokenization and Multi-Token Prediction
by: Wang, Yuhao, et al.
Published: (2025)

Cross-Modal Coreference Alignment: Enabling Reliable Information Transfer in Omni-LLMs
by: Liu, Hongcheng, et al.
Published: (2026)

Automatic Interactive Evaluation for Large Language Models with State Aware Patient Simulator
by: Liao, Yusheng, et al.
Published: (2024)

When Seeing Is not Enough: Revealing the Limits of Active Reasoning in MLLMs
by: Liu, Hongcheng, et al.
Published: (2025)

M2K-VDG: Model-Adaptive Multimodal Knowledge Anchor Enhanced Video-grounded Dialogue Generation
by: Liu, Hongcheng, et al.
Published: (2024)

M$^3$AV: A Multimodal, Multigenre, and Multipurpose Audio-Visual Academic Lecture Dataset
by: Chen, Zhe, et al.
Published: (2024)

LibriSQA: A Novel Dataset and Framework for Spoken Question Answering with Large Language Models
by: Zhao, Zihan, et al.
Published: (2023)

DisfluencySpeech -- Single-Speaker Conversational Speech Dataset with Paralanguage
by: Wang, Kyra, et al.
Published: (2024)

LaSR: Context-Aware Speech Recognition via Latent Reasoning
by: Liu, Heyang, et al.
Published: (2026)

Towards an End-to-End Framework for Invasive Brain Signal Decoding with Large Language Models
by: Feng, Sheng, et al.
Published: (2024)

OI-Bench: An Option Injection Benchmark for Evaluating LLM Susceptibility to Directive Interference
by: Liou, Yow-Fu, et al.
Published: (2026)

Mobile-Bench: An Evaluation Benchmark for LLM-based Mobile Agents
by: Deng, Shihan, et al.
Published: (2024)

Measuring the Effect of Disfluency in Multilingual Knowledge Probing Benchmarks
by: Semenov, Kirill, et al.
Published: (2025)

HeteroRAG: A Heterogeneous Retrieval-Augmented Generation Framework for Medical Vision Language Tasks
by: Chen, Zhe, et al.
Published: (2025)

Boosting Disfluency Detection with Large Language Model as Disfluency Generator
by: Cheng, Zhenrong, et al.
Published: (2024)

Augmenting Automatic Speech Recognition Models with Disfluency Detection
by: Amann, Robin, et al.
Published: (2024)

CliMedBench: A Large-Scale Chinese Benchmark for Evaluating Medical Large Language Models in Clinical Scenarios
by: Ouyang, Zetian, et al.
Published: (2024)

SNS-Bench-VL: Benchmarking Multimodal Large Language Models in Social Networking Services
by: Guo, Hongcheng, et al.
Published: (2025)

Pet-Bench: Benchmarking the Abilities of Large Language Models as E-Pets in Social Network Services
by: Guo, Hongcheng, et al.
Published: (2025)

Do These LLM Benchmarks Agree? Fixing Benchmark Evaluation with BenchBench
by: Perlitz, Yotam, et al.
Published: (2024)

Speech Robust Bench: A Robustness Benchmark For Speech Recognition
by: Shah, Muhammad A., et al.
Published: (2024)

CryptoBench: A Dynamic Benchmark for Expert-Level Evaluation of LLM Agents in Cryptocurrency
by: Guo, Jiacheng, et al.
Published: (2025)

FedLLM-Bench: Realistic Benchmarks for Federated Learning of Large Language Models
by: Ye, Rui, et al.
Published: (2024)

Enhancing Naturalness in LLM-Generated Utterances through Disfluency Insertion
by: Hassan, Syed Zohaib, et al.
Published: (2024)

SpeechParaling-Bench: A Comprehensive Benchmark for Paralinguistic-Aware Speech Generation
by: Liu, Ruohan, et al.
Published: (2026)

Selecting Auxiliary Data via Neural Tangent Kernels for Low-Resource Domains
by: Wang, Pingjie, et al.
Published: (2025)

MTalk-Bench: Evaluating Speech-to-Speech Models in Multi-Turn Dialogues via Arena-style and Rubrics Protocols
by: Du, Yuhao, et al.
Published: (2025)

WildSpeech-Bench: Benchmarking End-to-End SpeechLLMs in the Wild
by: Zhang, Linhao, et al.
Published: (2025)

EchoMind: An Interrelated Multi-level Benchmark for Evaluating Empathetic Speech Language Models
by: Zhou, Li, et al.
Published: (2025)