Joint Training of Speaker Embedding Extractor, Speech and Overlap Detection for Diarization
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929575126106112 |
|---|---|
| author | Pálka, Petr Landini, Federico Klement, Dominik Diez, Mireia Silnova, Anna Delcroix, Marc Burget, Lukáš |
| author_facet | Pálka, Petr Landini, Federico Klement, Dominik Diez, Mireia Silnova, Anna Delcroix, Marc Burget, Lukáš |
| contents | In spite of the popularity of end-to-end diarization systems nowadays, modular systems comprised of voice activity detection (VAD), speaker embedding extraction plus clustering, and overlapped speech detection (OSD) plus handling still attain competitive performance in many conditions. However, one of the main drawbacks of modular systems is the need to run (and train) different modules independently. In this work, we propose an approach to jointly train a model to produce speaker embeddings, VAD and OSD simultaneously and reach competitive performance at a fraction of the inference time of a standard approach. Furthermore, the joint inference leads to a simplified overall pipeline which brings us one step closer to a unified clustering-based method that can be trained end-to-end towards a diarization-specific objective. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_02165 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Joint Training of Speaker Embedding Extractor, Speech and Overlap Detection for Diarization Pálka, Petr Landini, Federico Klement, Dominik Diez, Mireia Silnova, Anna Delcroix, Marc Burget, Lukáš Audio and Speech Processing Sound In spite of the popularity of end-to-end diarization systems nowadays, modular systems comprised of voice activity detection (VAD), speaker embedding extraction plus clustering, and overlapped speech detection (OSD) plus handling still attain competitive performance in many conditions. However, one of the main drawbacks of modular systems is the need to run (and train) different modules independently. In this work, we propose an approach to jointly train a model to produce speaker embeddings, VAD and OSD simultaneously and reach competitive performance at a fraction of the inference time of a standard approach. Furthermore, the joint inference leads to a simplified overall pipeline which brings us one step closer to a unified clustering-based method that can be trained end-to-end towards a diarization-specific objective. |
| title | Joint Training of Speaker Embedding Extractor, Speech and Overlap Detection for Diarization |
| topic | Audio and Speech Processing Sound |
| url | https://arxiv.org/abs/2411.02165 |