Better Pseudo-labeling with Multi-ASR Fusion and Error Correction by SpeechLLM
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916986503561216 |
|---|---|
| author | Prakash, Jeena Kumar, Blessingh Hacioglu, Kadri Sharma, Bidisha Gopalan, Sindhuja Chetlur, Malolan Venkatesan, Shankar Stolcke, Andreas |
| author_facet | Prakash, Jeena Kumar, Blessingh Hacioglu, Kadri Sharma, Bidisha Gopalan, Sindhuja Chetlur, Malolan Venkatesan, Shankar Stolcke, Andreas |
| contents | Automatic speech recognition (ASR) models rely on high-quality transcribed data for effective training. Generating pseudo-labels for large unlabeled audio datasets often relies on complex pipelines that combine multiple ASR outputs through multi-stage processing, leading to error propagation, information loss and disjoint optimization. We propose a unified multi-ASR prompt-driven framework using postprocessing by either textual or speech-based large language models (LLMs), replacing voting or other arbitration logic for reconciling the ensemble outputs. We perform a comparative study of multiple architectures with and without LLMs, showing significant improvements in transcription accuracy compared to traditional methods. Furthermore, we use the pseudo-labels generated by the various approaches to train semi-supervised ASR models for different datasets, again showing improved performance with textual and speechLLM transcriptions compared to baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_11089 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Better Pseudo-labeling with Multi-ASR Fusion and Error Correction by SpeechLLM Prakash, Jeena Kumar, Blessingh Hacioglu, Kadri Sharma, Bidisha Gopalan, Sindhuja Chetlur, Malolan Venkatesan, Shankar Stolcke, Andreas Audio and Speech Processing Artificial Intelligence Computation and Language Automatic speech recognition (ASR) models rely on high-quality transcribed data for effective training. Generating pseudo-labels for large unlabeled audio datasets often relies on complex pipelines that combine multiple ASR outputs through multi-stage processing, leading to error propagation, information loss and disjoint optimization. We propose a unified multi-ASR prompt-driven framework using postprocessing by either textual or speech-based large language models (LLMs), replacing voting or other arbitration logic for reconciling the ensemble outputs. We perform a comparative study of multiple architectures with and without LLMs, showing significant improvements in transcription accuracy compared to traditional methods. Furthermore, we use the pseudo-labels generated by the various approaches to train semi-supervised ASR models for different datasets, again showing improved performance with textual and speechLLM transcriptions compared to baselines. |
| title | Better Pseudo-labeling with Multi-ASR Fusion and Error Correction by SpeechLLM |
| topic | Audio and Speech Processing Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2506.11089 |