Beyond Transcription: Mechanistic Interpretability in ASR
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915456607059968 |
|---|---|
| author | Glazer, Neta Segal-Feldman, Yael Segev, Hilit Shamsian, Aviv Buchnick, Asaf Hetz, Gill Fetaya, Ethan Keshet, Joseph Navon, Aviv |
| author_facet | Glazer, Neta Segal-Feldman, Yael Segev, Hilit Shamsian, Aviv Buchnick, Asaf Hetz, Gill Fetaya, Ethan Keshet, Joseph Navon, Aviv |
| contents | Interpretability methods have recently gained significant attention, particularly in the context of large language models, enabling insights into linguistic representations, error detection, and model behaviors such as hallucinations and repetitions. However, these techniques remain underexplored in automatic speech recognition (ASR), despite their potential to advance both the performance and interpretability of ASR systems. In this work, we adapt and systematically apply established interpretability methods such as logit lens, linear probing, and activation patching, to examine how acoustic and semantic information evolves across layers in ASR systems. Our experiments reveal previously unknown internal dynamics, including specific encoder-decoder interactions responsible for repetition hallucinations and semantic biases encoded deep within acoustic representations. These insights demonstrate the benefits of extending and applying interpretability techniques to speech recognition, opening promising directions for future research on improving model transparency and robustness. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_15882 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Beyond Transcription: Mechanistic Interpretability in ASR Glazer, Neta Segal-Feldman, Yael Segev, Hilit Shamsian, Aviv Buchnick, Asaf Hetz, Gill Fetaya, Ethan Keshet, Joseph Navon, Aviv Sound Computation and Language Machine Learning Audio and Speech Processing Interpretability methods have recently gained significant attention, particularly in the context of large language models, enabling insights into linguistic representations, error detection, and model behaviors such as hallucinations and repetitions. However, these techniques remain underexplored in automatic speech recognition (ASR), despite their potential to advance both the performance and interpretability of ASR systems. In this work, we adapt and systematically apply established interpretability methods such as logit lens, linear probing, and activation patching, to examine how acoustic and semantic information evolves across layers in ASR systems. Our experiments reveal previously unknown internal dynamics, including specific encoder-decoder interactions responsible for repetition hallucinations and semantic biases encoded deep within acoustic representations. These insights demonstrate the benefits of extending and applying interpretability techniques to speech recognition, opening promising directions for future research on improving model transparency and robustness. |
| title | Beyond Transcription: Mechanistic Interpretability in ASR |
| topic | Sound Computation and Language Machine Learning Audio and Speech Processing |
| url | https://arxiv.org/abs/2508.15882 |