E2 TTS: Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTS
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912023744348160 |
|---|---|
| author | Eskimez, Sefik Emre Wang, Xiaofei Thakker, Manthan Li, Canrun Tsai, Chung-Hsien Xiao, Zhen Yang, Hemin Zhu, Zirun Tang, Min Tan, Xu Liu, Yanqing Zhao, Sheng Kanda, Naoyuki |
| author_facet | Eskimez, Sefik Emre Wang, Xiaofei Thakker, Manthan Li, Canrun Tsai, Chung-Hsien Xiao, Zhen Yang, Hemin Zhu, Zirun Tang, Min Tan, Xu Liu, Yanqing Zhao, Sheng Kanda, Naoyuki |
| contents | This paper introduces Embarrassingly Easy Text-to-Speech (E2 TTS), a fully non-autoregressive zero-shot text-to-speech system that offers human-level naturalness and state-of-the-art speaker similarity and intelligibility. In the E2 TTS framework, the text input is converted into a character sequence with filler tokens. The flow-matching-based mel spectrogram generator is then trained based on the audio infilling task. Unlike many previous works, it does not require additional components (e.g., duration model, grapheme-to-phoneme) or complex techniques (e.g., monotonic alignment search). Despite its simplicity, E2 TTS achieves state-of-the-art zero-shot TTS capabilities that are comparable to or surpass previous works, including Voicebox and NaturalSpeech 3. The simplicity of E2 TTS also allows for flexibility in the input representation. We propose several variants of E2 TTS to improve usability during inference. See https://aka.ms/e2tts/ for demo samples. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_18009 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | E2 TTS: Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTS Eskimez, Sefik Emre Wang, Xiaofei Thakker, Manthan Li, Canrun Tsai, Chung-Hsien Xiao, Zhen Yang, Hemin Zhu, Zirun Tang, Min Tan, Xu Liu, Yanqing Zhao, Sheng Kanda, Naoyuki Audio and Speech Processing Sound This paper introduces Embarrassingly Easy Text-to-Speech (E2 TTS), a fully non-autoregressive zero-shot text-to-speech system that offers human-level naturalness and state-of-the-art speaker similarity and intelligibility. In the E2 TTS framework, the text input is converted into a character sequence with filler tokens. The flow-matching-based mel spectrogram generator is then trained based on the audio infilling task. Unlike many previous works, it does not require additional components (e.g., duration model, grapheme-to-phoneme) or complex techniques (e.g., monotonic alignment search). Despite its simplicity, E2 TTS achieves state-of-the-art zero-shot TTS capabilities that are comparable to or surpass previous works, including Voicebox and NaturalSpeech 3. The simplicity of E2 TTS also allows for flexibility in the input representation. We propose several variants of E2 TTS to improve usability during inference. See https://aka.ms/e2tts/ for demo samples. |
| title | E2 TTS: Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTS |
| topic | Audio and Speech Processing Sound |
| url | https://arxiv.org/abs/2406.18009 |