Efficient Speech Enhancement via Embeddings from Pre-trained Generative Audioencoders
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909648346415104 |
|---|---|
| author | Sun, Xingwei Dinkel, Heinrich Niu, Yadong Wang, Linzhang Zhang, Junbo Luan, Jian |
| author_facet | Sun, Xingwei Dinkel, Heinrich Niu, Yadong Wang, Linzhang Zhang, Junbo Luan, Jian |
| contents | Recent research has delved into speech enhancement (SE) approaches that leverage audio embeddings from pre-trained models, diverging from time-frequency masking or signal prediction techniques. This paper introduces an efficient and extensible SE method. Our approach involves initially extracting audio embeddings from noisy speech using a pre-trained audioencoder, which are then denoised by a compact encoder network. Subsequently, a vocoder synthesizes the clean speech from denoised embeddings. An ablation study substantiates the parameter efficiency of the denoise encoder with a pre-trained audioencoder and vocoder. Experimental results on both speech enhancement and speaker fidelity demonstrate that our generative audioencoder-based SE system outperforms models utilizing discriminative audioencoders. Furthermore, subjective listening tests validate that our proposed system surpasses an existing state-of-the-art SE model in terms of perceptual quality. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_11514 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Efficient Speech Enhancement via Embeddings from Pre-trained Generative Audioencoders Sun, Xingwei Dinkel, Heinrich Niu, Yadong Wang, Linzhang Zhang, Junbo Luan, Jian Audio and Speech Processing Sound Recent research has delved into speech enhancement (SE) approaches that leverage audio embeddings from pre-trained models, diverging from time-frequency masking or signal prediction techniques. This paper introduces an efficient and extensible SE method. Our approach involves initially extracting audio embeddings from noisy speech using a pre-trained audioencoder, which are then denoised by a compact encoder network. Subsequently, a vocoder synthesizes the clean speech from denoised embeddings. An ablation study substantiates the parameter efficiency of the denoise encoder with a pre-trained audioencoder and vocoder. Experimental results on both speech enhancement and speaker fidelity demonstrate that our generative audioencoder-based SE system outperforms models utilizing discriminative audioencoders. Furthermore, subjective listening tests validate that our proposed system surpasses an existing state-of-the-art SE model in terms of perceptual quality. |
| title | Efficient Speech Enhancement via Embeddings from Pre-trained Generative Audioencoders |
| topic | Audio and Speech Processing Sound |
| url | https://arxiv.org/abs/2506.11514 |