XANE: eXplainable Acoustic Neural Embeddings
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911910544277504 |
|---|---|
| author | Dumpala, Sri Harsha Sharma, Dushyant Sastri, Chandramouli Shama Kruchinin, Stanislav Fosburgh, James Naylor, Patrick A. |
| author_facet | Dumpala, Sri Harsha Sharma, Dushyant Sastri, Chandramouli Shama Kruchinin, Stanislav Fosburgh, James Naylor, Patrick A. |
| contents | We present a novel method for extracting neural embeddings that model the background acoustics of a speech signal. The extracted embeddings are used to estimate specific parameters related to the background acoustic properties of the signal in a non-intrusive manner, which allows the embeddings to be explainable in terms of those parameters. We illustrate the value of these embeddings by performing clustering experiments on unseen test data and show that the proposed embeddings achieve a mean F1 score of 95.2\% for three different tasks, outperforming significantly the WavLM based signal embeddings. We also show that the proposed method can explain the embeddings by estimating 14 acoustic parameters characterizing the background acoustics, including reverberation and noise levels, overlapped speech detection, CODEC type detection and noise type detection with high accuracy and a real-time factor 17 times lower than an external baseline method. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_05199 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | XANE: eXplainable Acoustic Neural Embeddings Dumpala, Sri Harsha Sharma, Dushyant Sastri, Chandramouli Shama Kruchinin, Stanislav Fosburgh, James Naylor, Patrick A. Audio and Speech Processing Sound We present a novel method for extracting neural embeddings that model the background acoustics of a speech signal. The extracted embeddings are used to estimate specific parameters related to the background acoustic properties of the signal in a non-intrusive manner, which allows the embeddings to be explainable in terms of those parameters. We illustrate the value of these embeddings by performing clustering experiments on unseen test data and show that the proposed embeddings achieve a mean F1 score of 95.2\% for three different tasks, outperforming significantly the WavLM based signal embeddings. We also show that the proposed method can explain the embeddings by estimating 14 acoustic parameters characterizing the background acoustics, including reverberation and noise levels, overlapped speech detection, CODEC type detection and noise type detection with high accuracy and a real-time factor 17 times lower than an external baseline method. |
| title | XANE: eXplainable Acoustic Neural Embeddings |
| topic | Audio and Speech Processing Sound |
| url | https://arxiv.org/abs/2406.05199 |