Automatic Assessment of Oral Reading Accuracy for Reading Diagnostics
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2023
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914881950711808 |
|---|---|
| author | Molenaar, Bo Tejedor-Garcia, Cristian Strik, Helmer Cucchiarini, Catia |
| author_facet | Molenaar, Bo Tejedor-Garcia, Cristian Strik, Helmer Cucchiarini, Catia |
| contents | Automatic assessment of reading fluency using automatic speech recognition (ASR) holds great potential for early detection of reading difficulties and subsequent timely intervention. Precise assessment tools are required, especially for languages other than English. In this study, we evaluate six state-of-the-art ASR-based systems for automatically assessing Dutch oral reading accuracy using Kaldi and Whisper. Results show our most successful system reached substantial agreement with human evaluations (MCC = .63). The same system reached the highest correlation between forced decoding confidence scores and word correctness (r = .45). This system's language model (LM) consisted of manual orthographic transcriptions and reading prompts of the test data, which shows that including reading errors in the LM improves assessment performance. We discuss the implications for developing automatic assessment systems and identify possible avenues of future research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2306_03444 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Automatic Assessment of Oral Reading Accuracy for Reading Diagnostics Molenaar, Bo Tejedor-Garcia, Cristian Strik, Helmer Cucchiarini, Catia Computation and Language Sound Audio and Speech Processing Signal Processing Automatic assessment of reading fluency using automatic speech recognition (ASR) holds great potential for early detection of reading difficulties and subsequent timely intervention. Precise assessment tools are required, especially for languages other than English. In this study, we evaluate six state-of-the-art ASR-based systems for automatically assessing Dutch oral reading accuracy using Kaldi and Whisper. Results show our most successful system reached substantial agreement with human evaluations (MCC = .63). The same system reached the highest correlation between forced decoding confidence scores and word correctness (r = .45). This system's language model (LM) consisted of manual orthographic transcriptions and reading prompts of the test data, which shows that including reading errors in the LM improves assessment performance. We discuss the implications for developing automatic assessment systems and identify possible avenues of future research. |
| title | Automatic Assessment of Oral Reading Accuracy for Reading Diagnostics |
| topic | Computation and Language Sound Audio and Speech Processing Signal Processing |
| url | https://arxiv.org/abs/2306.03444 |