Layer-wise Analysis for Quality of Multilingual Synthesized Speech
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914023195279360 |
|---|---|
| author | Cooper, Erica Okamoto, Takuma Ohtani, Yamato Toda, Tomoki Kawai, Hisashi |
| author_facet | Cooper, Erica Okamoto, Takuma Ohtani, Yamato Toda, Tomoki Kawai, Hisashi |
| contents | While supervised quality predictors for synthesized speech have demonstrated strong correlations with human ratings, their requirement for in-domain labeled training data hinders their generalization ability to new domains. Unsupervised approaches based on pretrained self-supervised learning (SSL) based models and automatic speech recognition (ASR) models are a promising alternative; however, little is known about how these models encode information about speech quality. Towards the goal of better understanding how different aspects of speech quality are encoded in a multilingual setting, we present a layer-wise analysis of multilingual pretrained speech models based on reference modeling. We find that features extracted from early SSL layers show correlations with human ratings of synthesized speech, and later layers of ASR models can predict quality of non-neural systems as well as intelligibility. We also demonstrate the importance of using well-matched reference data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_04830 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Layer-wise Analysis for Quality of Multilingual Synthesized Speech Cooper, Erica Okamoto, Takuma Ohtani, Yamato Toda, Tomoki Kawai, Hisashi Audio and Speech Processing Sound While supervised quality predictors for synthesized speech have demonstrated strong correlations with human ratings, their requirement for in-domain labeled training data hinders their generalization ability to new domains. Unsupervised approaches based on pretrained self-supervised learning (SSL) based models and automatic speech recognition (ASR) models are a promising alternative; however, little is known about how these models encode information about speech quality. Towards the goal of better understanding how different aspects of speech quality are encoded in a multilingual setting, we present a layer-wise analysis of multilingual pretrained speech models based on reference modeling. We find that features extracted from early SSL layers show correlations with human ratings of synthesized speech, and later layers of ASR models can predict quality of non-neural systems as well as intelligibility. We also demonstrate the importance of using well-matched reference data. |
| title | Layer-wise Analysis for Quality of Multilingual Synthesized Speech |
| topic | Audio and Speech Processing Sound |
| url | https://arxiv.org/abs/2509.04830 |