Leveraging Joint Spectral and Spatial Learning with MAMBA for Multichannel Speech Enhancement
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929675811422208 |
|---|---|
| author | Ren, Wenze Wu, Haibin Lin, Yi-Cheng Chen, Xuanjun Chao, Rong Hung, Kuo-Hsuan Li, You-Jin Ting, Wen-Yuan Wang, Hsin-Min Tsao, Yu |
| author_facet | Ren, Wenze Wu, Haibin Lin, Yi-Cheng Chen, Xuanjun Chao, Rong Hung, Kuo-Hsuan Li, You-Jin Ting, Wen-Yuan Wang, Hsin-Min Tsao, Yu |
| contents | In multichannel speech enhancement, effectively capturing spatial and spectral information across different microphones is crucial for noise reduction. Traditional methods, such as CNN or LSTM, attempt to model the temporal dynamics of full-band and sub-band spectral and spatial features. However, these approaches face limitations in fully modeling complex temporal dependencies, especially in dynamic acoustic environments. To overcome these challenges, we modify the current advanced model McNet by introducing an improved version of Mamba, a state-space model, and further propose MCMamba. MCMamba has been completely reengineered to integrate full-band and narrow-band spatial information with sub-band and full-band spectral features, providing a more comprehensive approach to modeling spatial and spectral information. Our experimental results demonstrate that MCMamba significantly improves the modeling of spatial and spectral features in multichannel speech enhancement, outperforming McNet and achieving state-of-the-art performance on the CHiME-3 dataset. Additionally, we find that Mamba performs exceptionally well in modeling spectral information. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_10376 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Leveraging Joint Spectral and Spatial Learning with MAMBA for Multichannel Speech Enhancement Ren, Wenze Wu, Haibin Lin, Yi-Cheng Chen, Xuanjun Chao, Rong Hung, Kuo-Hsuan Li, You-Jin Ting, Wen-Yuan Wang, Hsin-Min Tsao, Yu Audio and Speech Processing Sound In multichannel speech enhancement, effectively capturing spatial and spectral information across different microphones is crucial for noise reduction. Traditional methods, such as CNN or LSTM, attempt to model the temporal dynamics of full-band and sub-band spectral and spatial features. However, these approaches face limitations in fully modeling complex temporal dependencies, especially in dynamic acoustic environments. To overcome these challenges, we modify the current advanced model McNet by introducing an improved version of Mamba, a state-space model, and further propose MCMamba. MCMamba has been completely reengineered to integrate full-band and narrow-band spatial information with sub-band and full-band spectral features, providing a more comprehensive approach to modeling spatial and spectral information. Our experimental results demonstrate that MCMamba significantly improves the modeling of spatial and spectral features in multichannel speech enhancement, outperforming McNet and achieving state-of-the-art performance on the CHiME-3 dataset. Additionally, we find that Mamba performs exceptionally well in modeling spectral information. |
| title | Leveraging Joint Spectral and Spatial Learning with MAMBA for Multichannel Speech Enhancement |
| topic | Audio and Speech Processing Sound |
| url | https://arxiv.org/abs/2409.10376 |