Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2303.03849 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913660246425600 |
|---|---|
| author | Boeddeker, Christoph Subramanian, Aswin Shanmugam Wichern, Gordon Haeb-Umbach, Reinhold Roux, Jonathan Le |
| author_facet | Boeddeker, Christoph Subramanian, Aswin Shanmugam Wichern, Gordon Haeb-Umbach, Reinhold Roux, Jonathan Le |
| contents | Since diarization and source separation of meeting data are closely related tasks, we here propose an approach to perform the two objectives jointly. It builds upon the target-speaker voice activity detection (TS-VAD) diarization approach, which assumes that initial speaker embeddings are available. We replace the final combined speaker activity estimation network of TS-VAD with a network that produces speaker activity estimates at a time-frequency resolution. Those act as masks for source extraction, either via masking or via beamforming. The technique can be applied both for single-channel and multi-channel input and, in both cases, achieves a new state-of-the-art word error rate (WER) on the LibriCSS meeting data recognition task. We further compute speaker-aware and speaker-agnostic WERs to isolate the contribution of diarization errors to the overall WER performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2303_03849 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | TS-SEP: Joint Diarization and Separation Conditioned on Estimated Speaker Embeddings Boeddeker, Christoph Subramanian, Aswin Shanmugam Wichern, Gordon Haeb-Umbach, Reinhold Roux, Jonathan Le Audio and Speech Processing Sound Since diarization and source separation of meeting data are closely related tasks, we here propose an approach to perform the two objectives jointly. It builds upon the target-speaker voice activity detection (TS-VAD) diarization approach, which assumes that initial speaker embeddings are available. We replace the final combined speaker activity estimation network of TS-VAD with a network that produces speaker activity estimates at a time-frequency resolution. Those act as masks for source extraction, either via masking or via beamforming. The technique can be applied both for single-channel and multi-channel input and, in both cases, achieves a new state-of-the-art word error rate (WER) on the LibriCSS meeting data recognition task. We further compute speaker-aware and speaker-agnostic WERs to isolate the contribution of diarization errors to the overall WER performance. |
| title | TS-SEP: Joint Diarization and Separation Conditioned on Estimated Speaker Embeddings |
| topic | Audio and Speech Processing Sound |
| url | https://arxiv.org/abs/2303.03849 |