Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2604.18249 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910148793991168 |
|---|---|
| author | Herron, Felix Hjuler, Maja Rossato, Solange Allauzen, Alexandre Portet, François |
| author_facet | Herron, Felix Hjuler, Maja Rossato, Solange Allauzen, Alexandre Portet, François |
| contents | Speech encoder models are known to model members of some speaker groups (SGs) better than others. However, there has been little work in establishing why this occurs on a technological level. To our knowledge, we present the first layerwise fairness analysis of pretrained self-supervised speech encoder models (S3Ms), probing each embedding layer for speaker identification (SID) automatic speech recognition (ASR). We find S3Ms produce embeddings biased against certain SGs for both tasks, starting at the very first latent layers. Furthermore, we find opposite patterns of layerwise bias for SID vs ASR for all models in our study: SID bias is minimized in layers that minimize overall SID error; on the other hand, ASR bias is maximized in layers that minimize overall ASR error. The inverse bias/error relationship for ASR is unaffected when probing S3Ms that are finetuned for ASR, suggesting SG-level bias is established during pretraining and is difficult to remove. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_18249 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Where Do Self-Supervised Speech Models Become Unfair? Herron, Felix Hjuler, Maja Rossato, Solange Allauzen, Alexandre Portet, François Computation and Language Speech encoder models are known to model members of some speaker groups (SGs) better than others. However, there has been little work in establishing why this occurs on a technological level. To our knowledge, we present the first layerwise fairness analysis of pretrained self-supervised speech encoder models (S3Ms), probing each embedding layer for speaker identification (SID) automatic speech recognition (ASR). We find S3Ms produce embeddings biased against certain SGs for both tasks, starting at the very first latent layers. Furthermore, we find opposite patterns of layerwise bias for SID vs ASR for all models in our study: SID bias is minimized in layers that minimize overall SID error; on the other hand, ASR bias is maximized in layers that minimize overall ASR error. The inverse bias/error relationship for ASR is unaffected when probing S3Ms that are finetuned for ASR, suggesting SG-level bias is established during pretraining and is difficult to remove. |
| title | Where Do Self-Supervised Speech Models Become Unfair? |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2604.18249 |