Learning Robust Spatial Representations from Binaural Audio through Feature Distillation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916923701198848 |
|---|---|
| author | Bovbjerg, Holger Severin Østergaard, Jan Jensen, Jesper Watanabe, Shinji Tan, Zheng-Hua |
| author_facet | Bovbjerg, Holger Severin Østergaard, Jan Jensen, Jesper Watanabe, Shinji Tan, Zheng-Hua |
| contents | Recently, deep representation learning has shown strong performance in multiple audio tasks. However, its use for learning spatial representations from multichannel audio is underexplored. We investigate the use of a pretraining stage based on feature distillation to learn a robust spatial representation of binaural speech without the need for data labels. In this framework, spatial features are computed from clean binaural speech samples to form prediction labels. These clean features are then predicted from corresponding augmented speech using a neural network. After pretraining, we throw away the spatial feature predictor and use the learned encoder weights to initialize a DoA estimation model which we fine-tune for DoA estimation. Our experiments demonstrate that the pretrained models show improved performance in noisy and reverberant environments after fine-tuning for direction-of-arrival estimation, when compared to fully supervised models and classic signal processing methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_20914 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Learning Robust Spatial Representations from Binaural Audio through Feature Distillation Bovbjerg, Holger Severin Østergaard, Jan Jensen, Jesper Watanabe, Shinji Tan, Zheng-Hua Sound Machine Learning Audio and Speech Processing 68T10 I.2.6 Recently, deep representation learning has shown strong performance in multiple audio tasks. However, its use for learning spatial representations from multichannel audio is underexplored. We investigate the use of a pretraining stage based on feature distillation to learn a robust spatial representation of binaural speech without the need for data labels. In this framework, spatial features are computed from clean binaural speech samples to form prediction labels. These clean features are then predicted from corresponding augmented speech using a neural network. After pretraining, we throw away the spatial feature predictor and use the learned encoder weights to initialize a DoA estimation model which we fine-tune for DoA estimation. Our experiments demonstrate that the pretrained models show improved performance in noisy and reverberant environments after fine-tuning for direction-of-arrival estimation, when compared to fully supervised models and classic signal processing methods. |
| title | Learning Robust Spatial Representations from Binaural Audio through Feature Distillation |
| topic | Sound Machine Learning Audio and Speech Processing 68T10 I.2.6 |
| url | https://arxiv.org/abs/2508.20914 |