AudioSAE: Towards Understanding of Audio-Processing Models with Sparse AutoEncoders
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915779599925248 |
|---|---|
| author | Aparin, Georgii Sadekova, Tasnima Rukhovich, Alexey Yermekova, Assel Kushnareva, Laida Popov, Vadim Kuznetsov, Kristian Piontkovskaya, Irina |
| author_facet | Aparin, Georgii Sadekova, Tasnima Rukhovich, Alexey Yermekova, Assel Kushnareva, Laida Popov, Vadim Kuznetsov, Kristian Piontkovskaya, Irina |
| contents | Sparse Autoencoders (SAEs) are powerful tools for interpreting neural representations, yet their use in audio remains underexplored. We train SAEs across all encoder layers of Whisper and HuBERT, provide an extensive evaluation of their stability, interpretability, and show their practical utility. Over 50% of the features remain consistent across random seeds, and reconstruction quality is preserved. SAE features capture general acoustic and semantic information as well as specific events, including environmental noises and paralinguistic sounds (e.g. laughter, whispering) and disentangle them effectively, requiring removal of only 19-27% of features to erase a concept. Feature steering reduces Whisper's false speech detections by 70% with negligible WER increase, demonstrating real-world applicability. Finally, we find SAE features correlated with human EEG activity during speech perception, indicating alignment with human neural processing. The code and checkpoints are available at https://github.com/audiosae/audiosae_demo. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_05027 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | AudioSAE: Towards Understanding of Audio-Processing Models with Sparse AutoEncoders Aparin, Georgii Sadekova, Tasnima Rukhovich, Alexey Yermekova, Assel Kushnareva, Laida Popov, Vadim Kuznetsov, Kristian Piontkovskaya, Irina Sound Artificial Intelligence Sparse Autoencoders (SAEs) are powerful tools for interpreting neural representations, yet their use in audio remains underexplored. We train SAEs across all encoder layers of Whisper and HuBERT, provide an extensive evaluation of their stability, interpretability, and show their practical utility. Over 50% of the features remain consistent across random seeds, and reconstruction quality is preserved. SAE features capture general acoustic and semantic information as well as specific events, including environmental noises and paralinguistic sounds (e.g. laughter, whispering) and disentangle them effectively, requiring removal of only 19-27% of features to erase a concept. Feature steering reduces Whisper's false speech detections by 70% with negligible WER increase, demonstrating real-world applicability. Finally, we find SAE features correlated with human EEG activity during speech perception, indicating alignment with human neural processing. The code and checkpoints are available at https://github.com/audiosae/audiosae_demo. |
| title | AudioSAE: Towards Understanding of Audio-Processing Models with Sparse AutoEncoders |
| topic | Sound Artificial Intelligence |
| url | https://arxiv.org/abs/2602.05027 |