HiFi-Glot: High-Fidelity Neural Formant Synthesis with Differentiable Resonant Filters
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917274917535744 |
|---|---|
| author | Gu, Yicheng Zarazaga, Pablo Pérez Wang, Chaoren Wu, Zhizheng Malisz, Zofia Henter, Gustav Eje Juvela, Lauri |
| author_facet | Gu, Yicheng Zarazaga, Pablo Pérez Wang, Chaoren Wu, Zhizheng Malisz, Zofia Henter, Gustav Eje Juvela, Lauri |
| contents | Formant synthesis aims to generate speech with controllable formant structures, enabling precise control of vocal resonance and phonetic features. However, while existing formant synthesis approaches enable precise formant manipulation, they often yield an impoverished speech signal by failing to capture the complex co-occurring acoustic cues essential for naturalness. To address this issue, this letter presents HiFi-Glot, an end-to-end neural formant synthesis system that achieves both precise formant control and high-fidelity speech synthesis. Specifically, the proposed model adopts a source--filter architecture inspired by classical formant synthesis, where a neural vocoder generates the glottal excitation signal, and differentiable resonant filters model the formants to produce the speech waveform. Experiment results demonstrate that our proposed HiFi-Glot model can generate speech with higher perceptual quality and naturalness while exhibiting a more precise control over formant frequencies, outperforming industry-standard formant manipulation tools such as Praat. Code, checkpoints, and representative audio samples are available at https://www.yichenggu.com/HiFi-Glot/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_14823 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | HiFi-Glot: High-Fidelity Neural Formant Synthesis with Differentiable Resonant Filters Gu, Yicheng Zarazaga, Pablo Pérez Wang, Chaoren Wu, Zhizheng Malisz, Zofia Henter, Gustav Eje Juvela, Lauri Sound Audio and Speech Processing Formant synthesis aims to generate speech with controllable formant structures, enabling precise control of vocal resonance and phonetic features. However, while existing formant synthesis approaches enable precise formant manipulation, they often yield an impoverished speech signal by failing to capture the complex co-occurring acoustic cues essential for naturalness. To address this issue, this letter presents HiFi-Glot, an end-to-end neural formant synthesis system that achieves both precise formant control and high-fidelity speech synthesis. Specifically, the proposed model adopts a source--filter architecture inspired by classical formant synthesis, where a neural vocoder generates the glottal excitation signal, and differentiable resonant filters model the formants to produce the speech waveform. Experiment results demonstrate that our proposed HiFi-Glot model can generate speech with higher perceptual quality and naturalness while exhibiting a more precise control over formant frequencies, outperforming industry-standard formant manipulation tools such as Praat. Code, checkpoints, and representative audio samples are available at https://www.yichenggu.com/HiFi-Glot/. |
| title | HiFi-Glot: High-Fidelity Neural Formant Synthesis with Differentiable Resonant Filters |
| topic | Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2409.14823 |