Do End-to-End Neural Diarization Attractors Need to Encode Speaker Characteristic Information?
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911925652160512 |
|---|---|
| author | Zhang, Lin Stafylakis, Themos Landini, Federico Diez, Mireia Silnova, Anna Burget, Lukáš |
| author_facet | Zhang, Lin Stafylakis, Themos Landini, Federico Diez, Mireia Silnova, Anna Burget, Lukáš |
| contents | In this paper, we apply the variational information bottleneck approach to end-to-end neural diarization with encoder-decoder attractors (EEND-EDA). This allows us to investigate what information is essential for the model. EEND-EDA utilizes attractors, vector representations of speakers in a conversation. Our analysis shows that, attractors do not necessarily have to contain speaker characteristic information. On the other hand, giving the attractors more freedom to allow them to encode some extra (possibly speaker-specific) information leads to small but consistent diarization performance improvements. Despite architectural differences in EEND systems, the notion of attractors and frame embeddings is common to most of them and not specific to EEND-EDA. We believe that the main conclusions of this work can apply to other variants of EEND. Thus, we hope this paper will be a valuable contribution to guide the community to make more informed decisions when designing new systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_19325 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Do End-to-End Neural Diarization Attractors Need to Encode Speaker Characteristic Information? Zhang, Lin Stafylakis, Themos Landini, Federico Diez, Mireia Silnova, Anna Burget, Lukáš Sound Audio and Speech Processing In this paper, we apply the variational information bottleneck approach to end-to-end neural diarization with encoder-decoder attractors (EEND-EDA). This allows us to investigate what information is essential for the model. EEND-EDA utilizes attractors, vector representations of speakers in a conversation. Our analysis shows that, attractors do not necessarily have to contain speaker characteristic information. On the other hand, giving the attractors more freedom to allow them to encode some extra (possibly speaker-specific) information leads to small but consistent diarization performance improvements. Despite architectural differences in EEND systems, the notion of attractors and frame embeddings is common to most of them and not specific to EEND-EDA. We believe that the main conclusions of this work can apply to other variants of EEND. Thus, we hope this paper will be a valuable contribution to guide the community to make more informed decisions when designing new systems. |
| title | Do End-to-End Neural Diarization Attractors Need to Encode Speaker Characteristic Information? |
| topic | Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2402.19325 |