On the Effects of Heterogeneous Data Sources on Speech-to-Text Foundation Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911916259016704 |
|---|---|
| author | Tian, Jinchuan Peng, Yifan Chen, William Choi, Kwanghee Livescu, Karen Watanabe, Shinji |
| author_facet | Tian, Jinchuan Peng, Yifan Chen, William Choi, Kwanghee Livescu, Karen Watanabe, Shinji |
| contents | The Open Whisper-style Speech Model (OWSM) series was introduced to achieve full transparency in building advanced speech-to-text (S2T) foundation models. To this end, OWSM models are trained on 25 public speech datasets, which are heterogeneous in multiple ways. In this study, we advance the OWSM series by introducing OWSM v3.2, which improves on prior models by investigating and addressing the impacts of this data heterogeneity. Our study begins with a detailed analysis of each dataset, from which we derive two key strategies: data filtering with proxy task to enhance data quality, and the incorporation of punctuation and true-casing using an open large language model (LLM). With all other configurations staying the same, OWSM v3.2 improves performance over the OWSM v3.1 baseline while using 15% less training data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_09282 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | On the Effects of Heterogeneous Data Sources on Speech-to-Text Foundation Models Tian, Jinchuan Peng, Yifan Chen, William Choi, Kwanghee Livescu, Karen Watanabe, Shinji Computation and Language Sound Audio and Speech Processing The Open Whisper-style Speech Model (OWSM) series was introduced to achieve full transparency in building advanced speech-to-text (S2T) foundation models. To this end, OWSM models are trained on 25 public speech datasets, which are heterogeneous in multiple ways. In this study, we advance the OWSM series by introducing OWSM v3.2, which improves on prior models by investigating and addressing the impacts of this data heterogeneity. Our study begins with a detailed analysis of each dataset, from which we derive two key strategies: data filtering with proxy task to enhance data quality, and the incorporation of punctuation and true-casing using an open large language model (LLM). With all other configurations staying the same, OWSM v3.2 improves performance over the OWSM v3.1 baseline while using 15% less training data. |
| title | On the Effects of Heterogeneous Data Sources on Speech-to-Text Foundation Models |
| topic | Computation and Language Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2406.09282 |