Performance evaluation of SLAM-ASR: The Good, the Bad, the Ugly, and the Way Forward
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866929683815202816 |
|---|---|
| author | Kumar, Shashi Thorbecke, Iuliia Burdisso, Sergio Villatoro-Tello, Esaú E, Manjunath K Hacioğlu, Kadri Rangappa, Pradeep Motlicek, Petr Ganapathiraju, Aravind Stolcke, Andreas |
| author_facet | Kumar, Shashi Thorbecke, Iuliia Burdisso, Sergio Villatoro-Tello, Esaú E, Manjunath K Hacioğlu, Kadri Rangappa, Pradeep Motlicek, Petr Ganapathiraju, Aravind Stolcke, Andreas |
| contents | Recent research has demonstrated that training a linear connector between speech foundation encoders and large language models (LLMs) enables this architecture to achieve strong ASR capabilities. Despite the impressive results, it remains unclear whether these simple approaches are robust enough across different scenarios and speech conditions, such as domain shifts and speech perturbations. In this paper, we address these questions by conducting various ablation experiments using a recent and widely adopted approach called SLAM-ASR. We present novel empirical findings that offer insights on how to effectively utilize the SLAM-ASR architecture across a wide range of settings. Our main findings indicate that SLAM-ASR exhibits poor performance in cross-domain evaluation settings. Additionally, speech perturbations on in-domain data, such as changes in speech rate or additive noise, can significantly degrade performance. Our findings offer critical insights for fine-tuning and configuring robust LLM-based ASR models, tailored to different data characteristics and computational resources. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_03866 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Performance evaluation of SLAM-ASR: The Good, the Bad, the Ugly, and the Way Forward Kumar, Shashi Thorbecke, Iuliia Burdisso, Sergio Villatoro-Tello, Esaú E, Manjunath K Hacioğlu, Kadri Rangappa, Pradeep Motlicek, Petr Ganapathiraju, Aravind Stolcke, Andreas Computation and Language Artificial Intelligence Sound Audio and Speech Processing Recent research has demonstrated that training a linear connector between speech foundation encoders and large language models (LLMs) enables this architecture to achieve strong ASR capabilities. Despite the impressive results, it remains unclear whether these simple approaches are robust enough across different scenarios and speech conditions, such as domain shifts and speech perturbations. In this paper, we address these questions by conducting various ablation experiments using a recent and widely adopted approach called SLAM-ASR. We present novel empirical findings that offer insights on how to effectively utilize the SLAM-ASR architecture across a wide range of settings. Our main findings indicate that SLAM-ASR exhibits poor performance in cross-domain evaluation settings. Additionally, speech perturbations on in-domain data, such as changes in speech rate or additive noise, can significantly degrade performance. Our findings offer critical insights for fine-tuning and configuring robust LLM-based ASR models, tailored to different data characteristics and computational resources. |
| title | Performance evaluation of SLAM-ASR: The Good, the Bad, the Ugly, and the Way Forward |
| topic | Computation and Language Artificial Intelligence Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2411.03866 |