A Comparison of Differential Performance Metrics for the Evaluation of Automatic Speaker Verification Fairness
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913334249390080 |
|---|---|
| author | Chouchane, Oubaida Busch, Christoph Galdi, Chiara Evans, Nicholas Todisco, Massimiliano |
| author_facet | Chouchane, Oubaida Busch, Christoph Galdi, Chiara Evans, Nicholas Todisco, Massimiliano |
| contents | When decisions are made and when personal data is treated by automated processes, there is an expectation of fairness -- that members of different demographic groups receive equitable treatment. This expectation applies to biometric systems such as automatic speaker verification (ASV). We present a comparison of three candidate fairness metrics and extend previous work performed for face recognition, by examining differential performance across a range of different ASV operating points. Results show that the Gini Aggregation Rate for Biometric Equitability (GARBE) is the only one which meets three functional fairness measure criteria. Furthermore, a comprehensive evaluation of the fairness and verification performance of five state-of-the-art ASV systems is also presented. Our findings reveal a nuanced trade-off between fairness and verification accuracy underscoring the complex interplay between system design, demographic inclusiveness, and verification reliability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_17810 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Comparison of Differential Performance Metrics for the Evaluation of Automatic Speaker Verification Fairness Chouchane, Oubaida Busch, Christoph Galdi, Chiara Evans, Nicholas Todisco, Massimiliano Audio and Speech Processing Sound When decisions are made and when personal data is treated by automated processes, there is an expectation of fairness -- that members of different demographic groups receive equitable treatment. This expectation applies to biometric systems such as automatic speaker verification (ASV). We present a comparison of three candidate fairness metrics and extend previous work performed for face recognition, by examining differential performance across a range of different ASV operating points. Results show that the Gini Aggregation Rate for Biometric Equitability (GARBE) is the only one which meets three functional fairness measure criteria. Furthermore, a comprehensive evaluation of the fairness and verification performance of five state-of-the-art ASV systems is also presented. Our findings reveal a nuanced trade-off between fairness and verification accuracy underscoring the complex interplay between system design, demographic inclusiveness, and verification reliability. |
| title | A Comparison of Differential Performance Metrics for the Evaluation of Automatic Speaker Verification Fairness |
| topic | Audio and Speech Processing Sound |
| url | https://arxiv.org/abs/2404.17810 |