The Voice Behind the Words: Quantifying Intersectional Bias in SpeechLLMs
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910057339289600 |
|---|---|
| author | Satish, Shree Harsha Bokkahalli Minixhofer, Christoph Teleki, Maria Caverlee, James Klejch, Ondřej Bell, Peter Henter, Gustav Eje Székely, Éva |
| author_facet | Satish, Shree Harsha Bokkahalli Minixhofer, Christoph Teleki, Maria Caverlee, James Klejch, Ondřej Bell, Peter Henter, Gustav Eje Székely, Éva |
| contents | Speech Large Language Models (SpeechLLMs) process spoken input directly, retaining cues such as accent and perceived gender that were previously removed in cascaded pipelines. This introduces speaker identity dependent variation in responses. We present a large-scale intersectional evaluation of accent and gender bias in three SpeechLLMs using 2,880 controlled interactions across six English accents and two gender presentations, keeping linguistic content constant through voice cloning. Using pointwise LLM-judge ratings, pairwise comparisons, and Best-Worst Scaling with human validation, we detect consistent disparities. Eastern European-accented speech receives lower helpfulness scores, particularly for female-presenting voices. The bias is implicit: responses remain polite but differ in helpfulness. While LLM judges capture the directional trend of these biases, human evaluators exhibit significantly higher sensitivity, uncovering sharper intersectional disparities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_16941 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | The Voice Behind the Words: Quantifying Intersectional Bias in SpeechLLMs Satish, Shree Harsha Bokkahalli Minixhofer, Christoph Teleki, Maria Caverlee, James Klejch, Ondřej Bell, Peter Henter, Gustav Eje Székely, Éva Audio and Speech Processing Computation and Language Sound Speech Large Language Models (SpeechLLMs) process spoken input directly, retaining cues such as accent and perceived gender that were previously removed in cascaded pipelines. This introduces speaker identity dependent variation in responses. We present a large-scale intersectional evaluation of accent and gender bias in three SpeechLLMs using 2,880 controlled interactions across six English accents and two gender presentations, keeping linguistic content constant through voice cloning. Using pointwise LLM-judge ratings, pairwise comparisons, and Best-Worst Scaling with human validation, we detect consistent disparities. Eastern European-accented speech receives lower helpfulness scores, particularly for female-presenting voices. The bias is implicit: responses remain polite but differ in helpfulness. While LLM judges capture the directional trend of these biases, human evaluators exhibit significantly higher sensitivity, uncovering sharper intersectional disparities. |
| title | The Voice Behind the Words: Quantifying Intersectional Bias in SpeechLLMs |
| topic | Audio and Speech Processing Computation and Language Sound |
| url | https://arxiv.org/abs/2603.16941 |