Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2402.00337 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917580266012672 |
|---|---|
| author | Togami, Masahito Valin, Jean-Marc Helwani, Karim Giri, Ritwik Isik, Umut Goodwin, Michael M. |
| author_facet | Togami, Masahito Valin, Jean-Marc Helwani, Karim Giri, Ritwik Isik, Umut Goodwin, Michael M. |
| contents | We introduce a real-time, multichannel speech enhancement algorithm which maintains the spatial cues of stereo recordings including two speech sources. Recognizing that each source has unique spatial information, our method utilizes a dual-path structure, ensuring the spatial cues remain unaffected during enhancement by applying source-specific common-band gain. This method also seamlessly integrates pretrained monaural speech enhancement, eliminating the need for retraining on stereo inputs. Source separation from stereo mixtures is achieved via spatial beamforming, with the steering vector for each source being adaptively updated using post-enhancement output signal. This ensures accurate tracking of the spatial information. The final stereo output is derived by merging the spatial images of the enhanced sources, with its efficacy not heavily reliant on the separation performance of the beamforming. The algorithm runs in real-time on 10-ms frames with a 40 ms of look-ahead. Evaluations reveal its effectiveness in enhancing speech and preserving spatial cues in both fully and sparsely overlapped mixtures. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_00337 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Real-time Stereo Speech Enhancement with Spatial-Cue Preservation based on Dual-Path Structure Togami, Masahito Valin, Jean-Marc Helwani, Karim Giri, Ritwik Isik, Umut Goodwin, Michael M. Audio and Speech Processing We introduce a real-time, multichannel speech enhancement algorithm which maintains the spatial cues of stereo recordings including two speech sources. Recognizing that each source has unique spatial information, our method utilizes a dual-path structure, ensuring the spatial cues remain unaffected during enhancement by applying source-specific common-band gain. This method also seamlessly integrates pretrained monaural speech enhancement, eliminating the need for retraining on stereo inputs. Source separation from stereo mixtures is achieved via spatial beamforming, with the steering vector for each source being adaptively updated using post-enhancement output signal. This ensures accurate tracking of the spatial information. The final stereo output is derived by merging the spatial images of the enhanced sources, with its efficacy not heavily reliant on the separation performance of the beamforming. The algorithm runs in real-time on 10-ms frames with a 40 ms of look-ahead. Evaluations reveal its effectiveness in enhancing speech and preserving spatial cues in both fully and sparsely overlapped mixtures. |
| title | Real-time Stereo Speech Enhancement with Spatial-Cue Preservation based on Dual-Path Structure |
| topic | Audio and Speech Processing |
| url | https://arxiv.org/abs/2402.00337 |