Noise-robust Speech Separation with Fast Generative Correction
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917690626539520 |
|---|---|
| author | Wang, Helin Villalba, Jesus Moro-Velazquez, Laureano Hai, Jiarui Thebaud, Thomas Dehak, Najim |
| author_facet | Wang, Helin Villalba, Jesus Moro-Velazquez, Laureano Hai, Jiarui Thebaud, Thomas Dehak, Najim |
| contents | Speech separation, the task of isolating multiple speech sources from a mixed audio signal, remains challenging in noisy environments. In this paper, we propose a generative correction method to enhance the output of a discriminative separator. By leveraging a generative corrector based on a diffusion model, we refine the separation process for single-channel mixture speech by removing noises and perceptually unnatural distortions. Furthermore, we optimize the generative model using a predictive loss to streamline the diffusion model's reverse process into a single step and rectify any associated errors by the reverse process. Our method achieves state-of-the-art performance on the in-domain Libri2Mix noisy dataset, and out-of-domain WSJ with a variety of noises, improving SI-SNR by 22-35% relative to SepFormer, demonstrating robustness and strong generalization capabilities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_07461 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Noise-robust Speech Separation with Fast Generative Correction Wang, Helin Villalba, Jesus Moro-Velazquez, Laureano Hai, Jiarui Thebaud, Thomas Dehak, Najim Audio and Speech Processing Speech separation, the task of isolating multiple speech sources from a mixed audio signal, remains challenging in noisy environments. In this paper, we propose a generative correction method to enhance the output of a discriminative separator. By leveraging a generative corrector based on a diffusion model, we refine the separation process for single-channel mixture speech by removing noises and perceptually unnatural distortions. Furthermore, we optimize the generative model using a predictive loss to streamline the diffusion model's reverse process into a single step and rectify any associated errors by the reverse process. Our method achieves state-of-the-art performance on the in-domain Libri2Mix noisy dataset, and out-of-domain WSJ with a variety of noises, improving SI-SNR by 22-35% relative to SepFormer, demonstrating robustness and strong generalization capabilities. |
| title | Noise-robust Speech Separation with Fast Generative Correction |
| topic | Audio and Speech Processing |
| url | https://arxiv.org/abs/2406.07461 |