Array Geometry-Robust Attention-Based Neural Beamformer for Moving Speakers
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911919662694400 |
|---|---|
| author | Tammen, Marvin Ochiai, Tsubasa Delcroix, Marc Nakatani, Tomohiro Araki, Shoko Doclo, Simon |
| author_facet | Tammen, Marvin Ochiai, Tsubasa Delcroix, Marc Nakatani, Tomohiro Araki, Shoko Doclo, Simon |
| contents | Although mask-based beamforming is a powerful speech enhancement approach, it often requires manual parameter tuning to handle moving speakers. Recently, this approach was augmented with an attention-based spatial covariance matrix aggregator (ASA) module, enabling accurate tracking of moving speakers without manual tuning. However, the deep neural network model used in this module is limited to specific microphone arrays, necessitating a different model for varying channel permutations, numbers, or geometries. To improve the robustness of the ASA module against such variations, in this paper we investigate three approaches: training with random channel configurations, employing the transform-average-concatenate method to process multi-channel input features, and utilizing robust input features. Our experiments on the CHiME-3 and DEMAND datasets show that these approaches enable the ASA-augmented beamformer to track moving speakers across different microphone arrays unseen in training. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_03058 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Array Geometry-Robust Attention-Based Neural Beamformer for Moving Speakers Tammen, Marvin Ochiai, Tsubasa Delcroix, Marc Nakatani, Tomohiro Araki, Shoko Doclo, Simon Audio and Speech Processing Sound Although mask-based beamforming is a powerful speech enhancement approach, it often requires manual parameter tuning to handle moving speakers. Recently, this approach was augmented with an attention-based spatial covariance matrix aggregator (ASA) module, enabling accurate tracking of moving speakers without manual tuning. However, the deep neural network model used in this module is limited to specific microphone arrays, necessitating a different model for varying channel permutations, numbers, or geometries. To improve the robustness of the ASA module against such variations, in this paper we investigate three approaches: training with random channel configurations, employing the transform-average-concatenate method to process multi-channel input features, and utilizing robust input features. Our experiments on the CHiME-3 and DEMAND datasets show that these approaches enable the ASA-augmented beamformer to track moving speakers across different microphone arrays unseen in training. |
| title | Array Geometry-Robust Attention-Based Neural Beamformer for Moving Speakers |
| topic | Audio and Speech Processing Sound |
| url | https://arxiv.org/abs/2402.03058 |