VoiceTailor: Lightweight Plug-In Adapter for Diffusion-Based Personalized Text-to-Speech
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912004387635200 |
|---|---|
| author | Kim, Heeseung Lee, Sang-gil Yeom, Jiheum Lee, Che Hyun Kim, Sungwon Yoon, Sungroh |
| author_facet | Kim, Heeseung Lee, Sang-gil Yeom, Jiheum Lee, Che Hyun Kim, Sungwon Yoon, Sungroh |
| contents | We propose VoiceTailor, a parameter-efficient speaker-adaptive text-to-speech (TTS) system, by equipping a pre-trained diffusion-based TTS model with a personalized adapter. VoiceTailor identifies pivotal modules that benefit from the adapter based on a weight change ratio analysis. We utilize Low-Rank Adaptation (LoRA) as a parameter-efficient adaptation method and incorporate the adapter into pivotal modules of the pre-trained diffusion decoder. To achieve powerful adaptation performance with few parameters, we explore various guidance techniques for speaker adaptation and investigate the best strategies to strengthen speaker information. VoiceTailor demonstrates comparable speaker adaptation performance to existing adaptive TTS models by fine-tuning only 0.25\% of the total parameters. VoiceTailor shows strong robustness when adapting to a wide range of real-world speakers, as shown in the demo. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_14739 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | VoiceTailor: Lightweight Plug-In Adapter for Diffusion-Based Personalized Text-to-Speech Kim, Heeseung Lee, Sang-gil Yeom, Jiheum Lee, Che Hyun Kim, Sungwon Yoon, Sungroh Sound Audio and Speech Processing We propose VoiceTailor, a parameter-efficient speaker-adaptive text-to-speech (TTS) system, by equipping a pre-trained diffusion-based TTS model with a personalized adapter. VoiceTailor identifies pivotal modules that benefit from the adapter based on a weight change ratio analysis. We utilize Low-Rank Adaptation (LoRA) as a parameter-efficient adaptation method and incorporate the adapter into pivotal modules of the pre-trained diffusion decoder. To achieve powerful adaptation performance with few parameters, we explore various guidance techniques for speaker adaptation and investigate the best strategies to strengthen speaker information. VoiceTailor demonstrates comparable speaker adaptation performance to existing adaptive TTS models by fine-tuning only 0.25\% of the total parameters. VoiceTailor shows strong robustness when adapting to a wide range of real-world speakers, as shown in the demo. |
| title | VoiceTailor: Lightweight Plug-In Adapter for Diffusion-Based Personalized Text-to-Speech |
| topic | Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2408.14739 |