An Effective Automated Speaking Assessment Approach to Mitigating Data Scarcity and Imbalanced Distribution
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909518481326080 |
|---|---|
| author | Lo, Tien-Hong Chao, Fu-An Wu, Tzu-I Sung, Yao-Ting Chen, Berlin |
| author_facet | Lo, Tien-Hong Chao, Fu-An Wu, Tzu-I Sung, Yao-Ting Chen, Berlin |
| contents | Automated speaking assessment (ASA) typically involves automatic speech recognition (ASR) and hand-crafted feature extraction from the ASR transcript of a learner's speech. Recently, self-supervised learning (SSL) has shown stellar performance compared to traditional methods. However, SSL-based ASA systems are faced with at least three data-related challenges: limited annotated data, uneven distribution of learner proficiency levels and non-uniform score intervals between different CEFR proficiency levels. To address these challenges, we explore the use of two novel modeling strategies: metric-based classification and loss reweighting, leveraging distinct SSL-based embedding features. Extensive experimental results on the ICNALE benchmark dataset suggest that our approach can outperform existing strong baselines by a sizable margin, achieving a significant improvement of more than 10% in CEFR prediction accuracy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_07575 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | An Effective Automated Speaking Assessment Approach to Mitigating Data Scarcity and Imbalanced Distribution Lo, Tien-Hong Chao, Fu-An Wu, Tzu-I Sung, Yao-Ting Chen, Berlin Sound Artificial Intelligence Audio and Speech Processing Automated speaking assessment (ASA) typically involves automatic speech recognition (ASR) and hand-crafted feature extraction from the ASR transcript of a learner's speech. Recently, self-supervised learning (SSL) has shown stellar performance compared to traditional methods. However, SSL-based ASA systems are faced with at least three data-related challenges: limited annotated data, uneven distribution of learner proficiency levels and non-uniform score intervals between different CEFR proficiency levels. To address these challenges, we explore the use of two novel modeling strategies: metric-based classification and loss reweighting, leveraging distinct SSL-based embedding features. Extensive experimental results on the ICNALE benchmark dataset suggest that our approach can outperform existing strong baselines by a sizable margin, achieving a significant improvement of more than 10% in CEFR prediction accuracy. |
| title | An Effective Automated Speaking Assessment Approach to Mitigating Data Scarcity and Imbalanced Distribution |
| topic | Sound Artificial Intelligence Audio and Speech Processing |
| url | https://arxiv.org/abs/2404.07575 |