Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2406.18679 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917707022073856 |
|---|---|
| author | Li, Xiang Govindan, Vivek Paturi, Rohit Srinivasan, Sundararajan |
| author_facet | Li, Xiang Govindan, Vivek Paturi, Rohit Srinivasan, Sundararajan |
| contents | End-to-end neural diarization (EEND) models offer significant improvements over traditional embedding-based Speaker Diarization (SD) approaches but falls short on generalizing to long-form audio with large number of speakers. EEND-vector-clustering method mitigates this by combining local EEND with global clustering of speaker embeddings from local windows, but this requires an additional speaker embedding framework alongside the EEND module. In this paper, we propose a novel framework applying EEND both locally and globally for long-form audio without separate speaker embeddings. This approach achieves significant relative DER reduction of 13% and 10% over the conventional 1-pass EEND on Callhome American English and RT03-CTS datasets respectively and marginal improvements over EEND-vector-clustering without the need for additional speaker embeddings. Furthermore, we discuss the computational complexity of our proposed framework and explore strategies for reducing processing times. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_18679 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Speakers Unembedded: Embedding-free Approach to Long-form Neural Diarization Li, Xiang Govindan, Vivek Paturi, Rohit Srinivasan, Sundararajan Audio and Speech Processing Artificial Intelligence Computation and Language Machine Learning End-to-end neural diarization (EEND) models offer significant improvements over traditional embedding-based Speaker Diarization (SD) approaches but falls short on generalizing to long-form audio with large number of speakers. EEND-vector-clustering method mitigates this by combining local EEND with global clustering of speaker embeddings from local windows, but this requires an additional speaker embedding framework alongside the EEND module. In this paper, we propose a novel framework applying EEND both locally and globally for long-form audio without separate speaker embeddings. This approach achieves significant relative DER reduction of 13% and 10% over the conventional 1-pass EEND on Callhome American English and RT03-CTS datasets respectively and marginal improvements over EEND-vector-clustering without the need for additional speaker embeddings. Furthermore, we discuss the computational complexity of our proposed framework and explore strategies for reducing processing times. |
| title | Speakers Unembedded: Embedding-free Approach to Long-form Neural Diarization |
| topic | Audio and Speech Processing Artificial Intelligence Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2406.18679 |