Exploring Speech Foundation Models for Speaker Diarization in Child-Adult Dyadic Interactions
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914832555442176 |
|---|---|
| author | Xu, Anfeng Huang, Kevin Feng, Tiantian Shen, Lue Tager-Flusberg, Helen Narayanan, Shrikanth |
| author_facet | Xu, Anfeng Huang, Kevin Feng, Tiantian Shen, Lue Tager-Flusberg, Helen Narayanan, Shrikanth |
| contents | Speech foundation models, trained on vast datasets, have opened unique opportunities in addressing challenging low-resource speech understanding, such as child speech. In this work, we explore the capabilities of speech foundation models on child-adult speaker diarization. We show that exemplary foundation models can achieve 39.5% and 62.3% relative reductions in Diarization Error Rate and Speaker Confusion Rate, respectively, compared to previous speaker diarization methods. In addition, we benchmark and evaluate the speaker diarization results of the speech foundation models with varying the input audio window size, speaker demographics, and training data ratio. Our results highlight promising pathways for understanding and adopting speech foundation models to facilitate child speech understanding. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_07890 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Exploring Speech Foundation Models for Speaker Diarization in Child-Adult Dyadic Interactions Xu, Anfeng Huang, Kevin Feng, Tiantian Shen, Lue Tager-Flusberg, Helen Narayanan, Shrikanth Audio and Speech Processing Computation and Language Machine Learning Speech foundation models, trained on vast datasets, have opened unique opportunities in addressing challenging low-resource speech understanding, such as child speech. In this work, we explore the capabilities of speech foundation models on child-adult speaker diarization. We show that exemplary foundation models can achieve 39.5% and 62.3% relative reductions in Diarization Error Rate and Speaker Confusion Rate, respectively, compared to previous speaker diarization methods. In addition, we benchmark and evaluate the speaker diarization results of the speech foundation models with varying the input audio window size, speaker demographics, and training data ratio. Our results highlight promising pathways for understanding and adopting speech foundation models to facilitate child speech understanding. |
| title | Exploring Speech Foundation Models for Speaker Diarization in Child-Adult Dyadic Interactions |
| topic | Audio and Speech Processing Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2406.07890 |