Mamba-based Segmentation Model for Speaker Diarization
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909344018202624 |
|---|---|
| author | Plaquet, Alexis Tawara, Naohiro Delcroix, Marc Horiguchi, Shota Ando, Atsushi Araki, Shoko |
| author_facet | Plaquet, Alexis Tawara, Naohiro Delcroix, Marc Horiguchi, Shota Ando, Atsushi Araki, Shoko |
| contents | Mamba is a newly proposed architecture which behaves like a recurrent neural network (RNN) with attention-like capabilities. These properties are promising for speaker diarization, as attention-based models have unsuitable memory requirements for long-form audio, and traditional RNN capabilities are too limited. In this paper, we propose to assess the potential of Mamba for diarization by comparing the state-of-the-art neural segmentation of the pyannote pipeline with our proposed Mamba-based variant. Mamba's stronger processing capabilities allow usage of longer local windows, which significantly improve diarization quality by making the speaker embedding extraction more reliable. We find Mamba to be a superior alternative to both traditional RNN and the tested attention-based model. Our proposed Mamba-based system achieves state-of-the-art performance on three widely used diarization datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_06459 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Mamba-based Segmentation Model for Speaker Diarization Plaquet, Alexis Tawara, Naohiro Delcroix, Marc Horiguchi, Shota Ando, Atsushi Araki, Shoko Sound Audio and Speech Processing Mamba is a newly proposed architecture which behaves like a recurrent neural network (RNN) with attention-like capabilities. These properties are promising for speaker diarization, as attention-based models have unsuitable memory requirements for long-form audio, and traditional RNN capabilities are too limited. In this paper, we propose to assess the potential of Mamba for diarization by comparing the state-of-the-art neural segmentation of the pyannote pipeline with our proposed Mamba-based variant. Mamba's stronger processing capabilities allow usage of longer local windows, which significantly improve diarization quality by making the speaker embedding extraction more reliable. We find Mamba to be a superior alternative to both traditional RNN and the tested attention-based model. Our proposed Mamba-based system achieves state-of-the-art performance on three widely used diarization datasets. |
| title | Mamba-based Segmentation Model for Speaker Diarization |
| topic | Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2410.06459 |