Diffusion-based Generative Modeling with Discriminative Guidance for Streamable Speech Enhancement
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929391987064832 |
|---|---|
| author | Li, Chenda Cornell, Samuele Watanabe, Shinji Qian, Yanmin |
| author_facet | Li, Chenda Cornell, Samuele Watanabe, Shinji Qian, Yanmin |
| contents | Diffusion-based generative models (DGMs) have recently attracted attention in speech enhancement research (SE) as previous works showed a remarkable generalization capability. However, DGMs are also computationally intensive, as they usually require many iterations in the reverse diffusion process (RDP), making them impractical for streaming SE systems. In this paper, we propose to use discriminative scores from discriminative models in the first steps of the RDP. These discriminative scores require only one forward pass with the discriminative model for multiple RDP steps, thus greatly reducing computations. This approach also allows for performance improvements. We show that we can trade off between generative and discriminative capabilities as the number of steps with the discriminative score increases. Furthermore, we propose a novel streamable time-domain generative model with an algorithmic latency of 50 ms, which has no significant performance degradation compared to offline models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_13471 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Diffusion-based Generative Modeling with Discriminative Guidance for Streamable Speech Enhancement Li, Chenda Cornell, Samuele Watanabe, Shinji Qian, Yanmin Audio and Speech Processing Sound Diffusion-based generative models (DGMs) have recently attracted attention in speech enhancement research (SE) as previous works showed a remarkable generalization capability. However, DGMs are also computationally intensive, as they usually require many iterations in the reverse diffusion process (RDP), making them impractical for streaming SE systems. In this paper, we propose to use discriminative scores from discriminative models in the first steps of the RDP. These discriminative scores require only one forward pass with the discriminative model for multiple RDP steps, thus greatly reducing computations. This approach also allows for performance improvements. We show that we can trade off between generative and discriminative capabilities as the number of steps with the discriminative score increases. Furthermore, we propose a novel streamable time-domain generative model with an algorithmic latency of 50 ms, which has no significant performance degradation compared to offline models. |
| title | Diffusion-based Generative Modeling with Discriminative Guidance for Streamable Speech Enhancement |
| topic | Audio and Speech Processing Sound |
| url | https://arxiv.org/abs/2406.13471 |