Performance of Objective Speech Quality Metrics on Languages Beyond Validation Data: A Study of Turkish and Korean
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916751873146880 |
|---|---|
| author | Perez, Javier de Groot, Dimme Martinez, Jorge |
| author_facet | Perez, Javier de Groot, Dimme Martinez, Jorge |
| contents | Objective speech quality measures are widely used to assess the performance of video conferencing platforms and telecommunication systems. They predict human-rated speech quality and are crucial for assessing the systems quality of experience. Despite the widespread use, the quality measures are developed on a limited set of languages. This can be problematic since the performance on unseen languages is consequently not guaranteed or even studied. Here we raise awareness to this issue by investigating the performance of two objective speech quality measures (PESQ and ViSQOL) on Turkish and Korean. Using English as baseline, we show that Turkish samples have significantly higher ViSQOL scores and that for Turkish male speakers the correlation between PESQ and ViSQOL is highest. These results highlight the need to explore biases across metrics and to develop a labeled speech quality dataset with a variety of languages. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_16616 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Performance of Objective Speech Quality Metrics on Languages Beyond Validation Data: A Study of Turkish and Korean Perez, Javier de Groot, Dimme Martinez, Jorge Audio and Speech Processing Objective speech quality measures are widely used to assess the performance of video conferencing platforms and telecommunication systems. They predict human-rated speech quality and are crucial for assessing the systems quality of experience. Despite the widespread use, the quality measures are developed on a limited set of languages. This can be problematic since the performance on unseen languages is consequently not guaranteed or even studied. Here we raise awareness to this issue by investigating the performance of two objective speech quality measures (PESQ and ViSQOL) on Turkish and Korean. Using English as baseline, we show that Turkish samples have significantly higher ViSQOL scores and that for Turkish male speakers the correlation between PESQ and ViSQOL is highest. These results highlight the need to explore biases across metrics and to develop a labeled speech quality dataset with a variety of languages. |
| title | Performance of Objective Speech Quality Metrics on Languages Beyond Validation Data: A Study of Turkish and Korean |
| topic | Audio and Speech Processing |
| url | https://arxiv.org/abs/2505.16616 |