MAD Speech: Measures of Acoustic Diversity of Speech
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929751984177152 |
|---|---|
| author | Futeral, Matthieu Agostinelli, Andrea Tagliasacchi, Marco Zeghidour, Neil Kharitonov, Eugene |
| author_facet | Futeral, Matthieu Agostinelli, Andrea Tagliasacchi, Marco Zeghidour, Neil Kharitonov, Eugene |
| contents | Generative spoken language models produce speech in a wide range of voices, prosody, and recording conditions, seemingly approaching the diversity of natural speech. However, the extent to which generated speech is acoustically diverse remains unclear due to a lack of appropriate metrics. We address this gap by developing lightweight metrics of acoustic diversity, which we collectively refer to as MAD Speech. We focus on measuring five facets of acoustic diversity: voice, gender, emotion, accent, and background noise. We construct the metrics as a composition of specialized, per-facet embedding models and an aggregation function that measures diversity within the embedding space. Next, we build a series of datasets with a priori known diversity preferences for each facet. Using these datasets, we demonstrate that our proposed metrics achieve a stronger agreement with the ground-truth diversity than baselines. Finally, we showcase the applicability of our proposed metrics across several real-life evaluation scenarios. MAD Speech is made publicly accessible. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_10419 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MAD Speech: Measures of Acoustic Diversity of Speech Futeral, Matthieu Agostinelli, Andrea Tagliasacchi, Marco Zeghidour, Neil Kharitonov, Eugene Audio and Speech Processing Computation and Language Generative spoken language models produce speech in a wide range of voices, prosody, and recording conditions, seemingly approaching the diversity of natural speech. However, the extent to which generated speech is acoustically diverse remains unclear due to a lack of appropriate metrics. We address this gap by developing lightweight metrics of acoustic diversity, which we collectively refer to as MAD Speech. We focus on measuring five facets of acoustic diversity: voice, gender, emotion, accent, and background noise. We construct the metrics as a composition of specialized, per-facet embedding models and an aggregation function that measures diversity within the embedding space. Next, we build a series of datasets with a priori known diversity preferences for each facet. Using these datasets, we demonstrate that our proposed metrics achieve a stronger agreement with the ground-truth diversity than baselines. Finally, we showcase the applicability of our proposed metrics across several real-life evaluation scenarios. MAD Speech is made publicly accessible. |
| title | MAD Speech: Measures of Acoustic Diversity of Speech |
| topic | Audio and Speech Processing Computation and Language |
| url | https://arxiv.org/abs/2404.10419 |