Are These Even Words? Quantifying the Gibberishness of Generative Speech Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918169498615808 |
|---|---|
| author | de Oliveira, Danilo Peer, Tal Rochdi, Jonas Gerkmann, Timo |
| author_facet | de Oliveira, Danilo Peer, Tal Rochdi, Jonas Gerkmann, Timo |
| contents | Significant research efforts are currently being dedicated to non-intrusive quality and intelligibility assessment, especially given how it enables curation of large scale datasets of in-the-wild speech data. However, with the increasing capabilities of generative models to synthesize high quality speech, new types of artifacts become relevant, such as generative hallucinations. While intrusive metrics are able to spot such sort of discrepancies from a reference signal, it is not clear how current non-intrusive methods react to high-quality phoneme confusions or, more extremely, gibberish speech. In this paper we explore how to factor in this aspect under a fully unsupervised setting by leveraging language models. Additionally, we publish a dataset of high-quality synthesized gibberish speech for further development of measures to assess implausible sentences in spoken language, alongside code for calculating scores from a variety of speech language models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_21317 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Are These Even Words? Quantifying the Gibberishness of Generative Speech Models de Oliveira, Danilo Peer, Tal Rochdi, Jonas Gerkmann, Timo Audio and Speech Processing Sound Significant research efforts are currently being dedicated to non-intrusive quality and intelligibility assessment, especially given how it enables curation of large scale datasets of in-the-wild speech data. However, with the increasing capabilities of generative models to synthesize high quality speech, new types of artifacts become relevant, such as generative hallucinations. While intrusive metrics are able to spot such sort of discrepancies from a reference signal, it is not clear how current non-intrusive methods react to high-quality phoneme confusions or, more extremely, gibberish speech. In this paper we explore how to factor in this aspect under a fully unsupervised setting by leveraging language models. Additionally, we publish a dataset of high-quality synthesized gibberish speech for further development of measures to assess implausible sentences in spoken language, alongside code for calculating scores from a variety of speech language models. |
| title | Are These Even Words? Quantifying the Gibberishness of Generative Speech Models |
| topic | Audio and Speech Processing Sound |
| url | https://arxiv.org/abs/2510.21317 |