Preset-Voice Matching for Privacy Regulated Speech-to-Speech Translation Systems
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913435090944000 |
|---|---|
| author | Platnick, Daniel Abdelnour, Bishoy Earl, Eamon Kumar, Rahul Rezaei, Zahra Tsangaris, Thomas Lagum, Faraj |
| author_facet | Platnick, Daniel Abdelnour, Bishoy Earl, Eamon Kumar, Rahul Rezaei, Zahra Tsangaris, Thomas Lagum, Faraj |
| contents | In recent years, there has been increased demand for speech-to-speech translation (S2ST) systems in industry settings. Although successfully commercialized, cloning-based S2ST systems expose their distributors to liabilities when misused by individuals and can infringe on personality rights when exploited by media organizations. This work proposes a regulated S2ST framework called Preset-Voice Matching (PVM). PVM removes cross-lingual voice cloning in S2ST by first matching the input voice to a similar prior consenting speaker voice in the target-language. With this separation, PVM avoids cloning the input speaker, ensuring PVM systems comply with regulations and reduce risk of misuse. Our results demonstrate PVM can significantly improve S2ST system run-time in multi-speaker settings and the naturalness of S2ST synthesized speech. To our knowledge, PVM is the first explicitly regulated S2ST framework leveraging similarly-matched preset-voices for dynamic S2ST tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_13153 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Preset-Voice Matching for Privacy Regulated Speech-to-Speech Translation Systems Platnick, Daniel Abdelnour, Bishoy Earl, Eamon Kumar, Rahul Rezaei, Zahra Tsangaris, Thomas Lagum, Faraj Computation and Language Cryptography and Security Machine Learning Sound Audio and Speech Processing In recent years, there has been increased demand for speech-to-speech translation (S2ST) systems in industry settings. Although successfully commercialized, cloning-based S2ST systems expose their distributors to liabilities when misused by individuals and can infringe on personality rights when exploited by media organizations. This work proposes a regulated S2ST framework called Preset-Voice Matching (PVM). PVM removes cross-lingual voice cloning in S2ST by first matching the input voice to a similar prior consenting speaker voice in the target-language. With this separation, PVM avoids cloning the input speaker, ensuring PVM systems comply with regulations and reduce risk of misuse. Our results demonstrate PVM can significantly improve S2ST system run-time in multi-speaker settings and the naturalness of S2ST synthesized speech. To our knowledge, PVM is the first explicitly regulated S2ST framework leveraging similarly-matched preset-voices for dynamic S2ST tasks. |
| title | Preset-Voice Matching for Privacy Regulated Speech-to-Speech Translation Systems |
| topic | Computation and Language Cryptography and Security Machine Learning Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2407.13153 |