Cover Image

Evaluation of LLMs in Speech is Often Flawed: Test Set Contamination in Large Language Models for Speech Recognition

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Tseng, Yuan, Parcollet, Titouan, van Dalen, Rogier, Zhang, Shucong, Bhattacharya, Sourav
Format:	Preprint
Published:	2025
Subjects:	Audio and Speech Processing Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Benchmarking Rotary Position Embeddings for Automatic Speech Recognition
by: Zhang, Shucong, et al.
Published: (2025)

SummaryMixing: A Linear-Complexity Alternative to Self-Attention for Speech Recognition and Understanding
by: Parcollet, Titouan, et al.
Published: (2023)

Loquacious Set: 25,000 Hours of Transcribed and Diverse English Speech Recognition Data for Research and Commercial Use
by: Parcollet, Titouan, et al.
Published: (2025)

Linear-Complexity Self-Supervised Learning for Speech Processing
by: Zhang, Shucong, et al.
Published: (2024)

Robust Unsupervised Adaptation of a Speech Recogniser Using Entropy Minimisation and Speaker Codes
by: van Dalen, Rogier C., et al.
Published: (2025)

Linear Time Complexity Conformers with SummaryMixing for Streaming Speech Recognition
by: Parcollet, Titouan, et al.
Published: (2024)

Streaming Speech-to-Text Translation with a SpeechLLM
by: Parcollet, Titouan, et al.
Published: (2026)

Analyzing Speech Unit Selection for Textless Speech-to-Speech Translation
by: Duret, Jarod, et al.
Published: (2024)

Less Forgetting for Better Generalization: Exploring Continual-learning Fine-tuning Methods for Speech Self-supervised Representations
by: Zaiem, Salah, et al.
Published: (2024)

Towards Early Prediction of Self-Supervised Speech Model Performance
by: Whetten, Ryan, et al.
Published: (2025)

A Study of Data Selection Strategies for Pre-training Self-Supervised Speech Models
by: Whetten, Ryan, et al.
Published: (2026)

Zero-resource Speech Translation and Recognition with LLMs
by: Mundnich, Karel, et al.
Published: (2024)

A Large Dataset of Spontaneous Speech with the Accent Spoken in São Paulo for Automatic Speech Recognition Evaluation
by: Lima, Rodrigo, et al.
Published: (2024)

Multi-stage Large Language Model Correction for Speech Recognition
by: Pu, Jie, et al.
Published: (2023)

Generating Data with Text-to-Speech and Large-Language Models for Conversational Speech Recognition
by: Cornell, Samuele, et al.
Published: (2024)

Codec2Vec: Self-Supervised Speech Representation Learning Using Neural Speech Codecs
by: Tseng, Wei-Cheng, et al.
Published: (2025)

SpeechColab Leaderboard: An Open-Source Platform for Automatic Speech Recognition Evaluation
by: Du, Jiayu, et al.
Published: (2024)

Evaluating Emotion Recognition in Spoken Language Models on Emotionally Incongruent Speech
by: Corrêa, Pedro, et al.
Published: (2025)

Chain of Correction for Full-text Speech Recognition with Large Language Models
by: Tang, Zhiyuan, et al.
Published: (2025)

Streaming Speech-to-Confusion Network Speech Recognition
by: Filimonov, Denis, et al.
Published: (2023)

Speech Recognition Rescoring with Large Speech-Text Foundation Models
by: Shivakumar, Prashanth Gurunath, et al.
Published: (2024)

Dolphin: A Large-Scale Automatic Speech Recognition Model for Eastern Languages
by: Meng, Yangyang, et al.
Published: (2025)

Full-text Error Correction for Chinese Speech Recognition with Large Language Model
by: Tang, Zhiyuan, et al.
Published: (2024)

Customizing Speech Recognition Model with Large Language Model Feedback
by: Ling, Shaoshi, et al.
Published: (2025)

Transducer-Llama: Integrating LLMs into Streamable Transducer-based Speech Recognition
by: Deng, Keqi, et al.
Published: (2024)

Speech Self-Supervised Representations Benchmarking: a Case for Larger Probing Heads
by: Zaiem, Salah, et al.
Published: (2023)

PAC: Pronunciation-Aware Contextualized Large Language Model-based Automatic Speech Recognition
by: Fu, Li, et al.
Published: (2025)

Transducer Consistency Regularization for Speech to Text Applications
by: Tseng, Cindy, et al.
Published: (2024)

Enhancing Speech Large Language Models through Reinforced Behavior Alignment
by: Liu, Yansong, et al.
Published: (2025)

SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models
by: Zhang, Xin, et al.
Published: (2023)

LI-TTA: Language Informed Test-Time Adaptation for Automatic Speech Recognition
by: Yoon, Eunseop, et al.
Published: (2024)

Inappropriate Pause Detection In Dysarthric Speech Using Large-Scale Speech Recognition
by: Lee, Jeehyun, et al.
Published: (2024)

Granary: Speech Recognition and Translation Dataset in 25 European Languages
by: Koluguri, Nithin Rao, et al.
Published: (2025)

Enhancing Indonesian Automatic Speech Recognition: Evaluating Multilingual Models with Diverse Speech Variabilities
by: Adila, Aulia, et al.
Published: (2024)

Improving Speech Emotion Recognition in Under-Resourced Languages via Speech-to-Speech Translation with Bootstrapping Data Selection
by: Lin, Hsi-Che, et al.
Published: (2024)

A Large-Scale Evaluation of Speech Foundation Models
by: Yang, Shu-wen, et al.
Published: (2024)

Speech-IFEval: Evaluating Instruction-Following and Quantifying Catastrophic Forgetting in Speech-Aware Language Models
by: Lu, Ke-Han, et al.
Published: (2025)

SMILE: Speech Meta In-Context Learning for Low-Resource Language Automatic Speech Recognition
by: Hsu, Ming-Hao, et al.
Published: (2024)

Pinyin Regularization in Error Correction for Chinese Speech Recognition with Large Language Models
by: Tang, Zhiyuan, et al.
Published: (2024)

Spoken Stereoset: On Evaluating Social Bias Toward Speaker in Speech Large Language Models
by: Lin, Yi-Cheng, et al.
Published: (2024)