Analyzing and Improving Speaker Similarity Assessment for Speech Synthesis
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913923716874240 |
|---|---|
| author | Carbonneau, Marc-André van Niekerk, Benjamin Seuté, Hugo Letendre, Jean-Philippe Kamper, Herman Zaïdi, Julian |
| author_facet | Carbonneau, Marc-André van Niekerk, Benjamin Seuté, Hugo Letendre, Jean-Philippe Kamper, Herman Zaïdi, Julian |
| contents | Modeling voice identity is challenging due to its multifaceted nature. In generative speech systems, identity is often assessed using automatic speaker verification (ASV) embeddings, designed for discrimination rather than characterizing identity. This paper investigates which aspects of a voice are captured in such representations. We find that widely used ASV embeddings focus mainly on static features like timbre and pitch range, while neglecting dynamic elements such as rhythm. We also identify confounding factors that compromise speaker similarity measurements and suggest mitigation strategies. To address these gaps, we propose U3D, a metric that evaluates speakers' dynamic rhythm patterns. This work contributes to the ongoing challenge of assessing speaker identity consistency in the context of ever-better voice cloning systems. We publicly release our code. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_02176 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Analyzing and Improving Speaker Similarity Assessment for Speech Synthesis Carbonneau, Marc-André van Niekerk, Benjamin Seuté, Hugo Letendre, Jean-Philippe Kamper, Herman Zaïdi, Julian Sound Computation and Language Machine Learning Audio and Speech Processing Modeling voice identity is challenging due to its multifaceted nature. In generative speech systems, identity is often assessed using automatic speaker verification (ASV) embeddings, designed for discrimination rather than characterizing identity. This paper investigates which aspects of a voice are captured in such representations. We find that widely used ASV embeddings focus mainly on static features like timbre and pitch range, while neglecting dynamic elements such as rhythm. We also identify confounding factors that compromise speaker similarity measurements and suggest mitigation strategies. To address these gaps, we propose U3D, a metric that evaluates speakers' dynamic rhythm patterns. This work contributes to the ongoing challenge of assessing speaker identity consistency in the context of ever-better voice cloning systems. We publicly release our code. |
| title | Analyzing and Improving Speaker Similarity Assessment for Speech Synthesis |
| topic | Sound Computation and Language Machine Learning Audio and Speech Processing |
| url | https://arxiv.org/abs/2507.02176 |