TalTech-IRIT-LIS Speaker and Language Diarization Systems for DISPLACE 2024
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913434528907264 |
|---|---|
| author | Kalda, Joonas Alumäe, Tanel Lebourdais, Martin Bredin, Hervé Baroudi, Séverin Marxer, Ricard |
| author_facet | Kalda, Joonas Alumäe, Tanel Lebourdais, Martin Bredin, Hervé Baroudi, Séverin Marxer, Ricard |
| contents | This paper describes the submissions of team TalTech-IRIT-LIS to the DISPLACE 2024 challenge. Our team participated in the speaker diarization and language diarization tracks of the challenge. In the speaker diarization track, our best submission was an ensemble of systems based on the pyannote.audio speaker diarization pipeline utilizing powerset training and our recently proposed PixIT method that performs joint diarization and speech separation. We improve upon PixIT by using the separation outputs for speaker embedding extraction. Our ensemble achieved a diarization error rate of 27.1% on the evaluation dataset. In the language diarization track, we fine-tuned a pre-trained Wav2Vec2-BERT language embedding model on in-domain data, and clustered short segments using AHC and VBx, based on similarity scores from LDA/PLDA. This led to a language diarization error rate of 27.6% on the evaluation data. Both results were ranked first in their respective challenge tracks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_12743 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | TalTech-IRIT-LIS Speaker and Language Diarization Systems for DISPLACE 2024 Kalda, Joonas Alumäe, Tanel Lebourdais, Martin Bredin, Hervé Baroudi, Séverin Marxer, Ricard Audio and Speech Processing This paper describes the submissions of team TalTech-IRIT-LIS to the DISPLACE 2024 challenge. Our team participated in the speaker diarization and language diarization tracks of the challenge. In the speaker diarization track, our best submission was an ensemble of systems based on the pyannote.audio speaker diarization pipeline utilizing powerset training and our recently proposed PixIT method that performs joint diarization and speech separation. We improve upon PixIT by using the separation outputs for speaker embedding extraction. Our ensemble achieved a diarization error rate of 27.1% on the evaluation dataset. In the language diarization track, we fine-tuned a pre-trained Wav2Vec2-BERT language embedding model on in-domain data, and clustered short segments using AHC and VBx, based on similarity scores from LDA/PLDA. This led to a language diarization error rate of 27.6% on the evaluation data. Both results were ranked first in their respective challenge tracks. |
| title | TalTech-IRIT-LIS Speaker and Language Diarization Systems for DISPLACE 2024 |
| topic | Audio and Speech Processing |
| url | https://arxiv.org/abs/2407.12743 |