The Unheard Alternative: Contrastive Explanations for Speech-to-Text Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911627408834560 |
|---|---|
| author | Conti, Lina Fucci, Dennis Gaido, Marco Negri, Matteo Wisniewski, Guillaume Bentivogli, Luisa |
| author_facet | Conti, Lina Fucci, Dennis Gaido, Marco Negri, Matteo Wisniewski, Guillaume Bentivogli, Luisa |
| contents | Contrastive explanations, which indicate why an AI system produced one output (the target) instead of another (the foil), are widely regarded in explainable AI as more informative and interpretable than standard explanations. However, obtaining such explanations for speech-to-text (S2T) generative models remains an open challenge. Drawing from feature attribution techniques, we propose the first method to obtain contrastive explanations in S2T by analyzing how parts of the input spectrogram influence the choice between alternative outputs. Through a case study on gender assignment in speech translation, we show that our method accurately identifies the audio features that drive the selection of one gender over another. By extending the scope of contrastive explanations to S2T, our work provides a foundation for better understanding S2T models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_26543 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | The Unheard Alternative: Contrastive Explanations for Speech-to-Text Models Conti, Lina Fucci, Dennis Gaido, Marco Negri, Matteo Wisniewski, Guillaume Bentivogli, Luisa Computation and Language Artificial Intelligence Contrastive explanations, which indicate why an AI system produced one output (the target) instead of another (the foil), are widely regarded in explainable AI as more informative and interpretable than standard explanations. However, obtaining such explanations for speech-to-text (S2T) generative models remains an open challenge. Drawing from feature attribution techniques, we propose the first method to obtain contrastive explanations in S2T by analyzing how parts of the input spectrogram influence the choice between alternative outputs. Through a case study on gender assignment in speech translation, we show that our method accurately identifies the audio features that drive the selection of one gender over another. By extending the scope of contrastive explanations to S2T, our work provides a foundation for better understanding S2T models. |
| title | The Unheard Alternative: Contrastive Explanations for Speech-to-Text Models |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2509.26543 |