A Novel Data Augmentation Approach for Automatic Speaking Assessment on Opinion Expressions
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866912581097095168 |
|---|---|
| author | Wang, Chung-Chun Lin, Jhen-Ke Lu, Hao-Chien Lin, Hong-Yun Chen, Berlin |
| author_facet | Wang, Chung-Chun Lin, Jhen-Ke Lu, Hao-Chien Lin, Hong-Yun Chen, Berlin |
| contents | Automated speaking assessment (ASA) on opinion expressions is often hampered by the scarcity of labeled recordings, which restricts prompt diversity and undermines scoring reliability. To address this challenge, we propose a novel training paradigm that leverages a large language models (LLM) to generate diverse responses of a given proficiency level, converts responses into synthesized speech via speaker-aware text-to-speech synthesis, and employs a dynamic importance loss to adaptively reweight training instances based on feature distribution differences between synthesized and real speech. Subsequently, a multimodal large language model integrates aligned textual features with speech signals to predict proficiency scores directly. Experiments conducted on the LTTC dataset show that our approach outperforms methods relying on real data or conventional augmentation, effectively mitigating low-resource constraints and enabling ASA on opinion expressions with cross-modal information. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_04077 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Novel Data Augmentation Approach for Automatic Speaking Assessment on Opinion Expressions Wang, Chung-Chun Lin, Jhen-Ke Lu, Hao-Chien Lin, Hong-Yun Chen, Berlin Computation and Language Sound Audio and Speech Processing Automated speaking assessment (ASA) on opinion expressions is often hampered by the scarcity of labeled recordings, which restricts prompt diversity and undermines scoring reliability. To address this challenge, we propose a novel training paradigm that leverages a large language models (LLM) to generate diverse responses of a given proficiency level, converts responses into synthesized speech via speaker-aware text-to-speech synthesis, and employs a dynamic importance loss to adaptively reweight training instances based on feature distribution differences between synthesized and real speech. Subsequently, a multimodal large language model integrates aligned textual features with speech signals to predict proficiency scores directly. Experiments conducted on the LTTC dataset show that our approach outperforms methods relying on real data or conventional augmentation, effectively mitigating low-resource constraints and enabling ASA on opinion expressions with cross-modal information. |
| title | A Novel Data Augmentation Approach for Automatic Speaking Assessment on Opinion Expressions |
| topic | Computation and Language Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2506.04077 |