Can you Remove the Downstream Model for Speaker Recognition with Self-Supervised Speech Features?
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911914908450816 |
|---|---|
| author | Aldeneh, Zakaria Higuchi, Takuya Jung, Jee-weon Seto, Skyler Likhomanenko, Tatiana Shum, Stephen Abdelaziz, Ahmed Hussen Watanabe, Shinji Theobald, Barry-John |
| author_facet | Aldeneh, Zakaria Higuchi, Takuya Jung, Jee-weon Seto, Skyler Likhomanenko, Tatiana Shum, Stephen Abdelaziz, Ahmed Hussen Watanabe, Shinji Theobald, Barry-John |
| contents | Self-supervised features are typically used in place of filter-bank features in speaker verification models. However, these models were originally designed to ingest filter-bank features as inputs, and thus, training them on top of self-supervised features assumes that both feature types require the same amount of learning for the task. In this work, we observe that pre-trained self-supervised speech features inherently include information required for downstream speaker verification task, and therefore, we can simplify the downstream model without sacrificing performance. To this end, we revisit the design of the downstream model for speaker verification using self-supervised features. We show that we can simplify the model to use 97.51% fewer parameters while achieving a 29.93% average improvement in performance on SUPERB. Consequently, we show that the simplified downstream model is more data efficient compared to baseline--it achieves better performance with only 60% of the training data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_00340 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Can you Remove the Downstream Model for Speaker Recognition with Self-Supervised Speech Features? Aldeneh, Zakaria Higuchi, Takuya Jung, Jee-weon Seto, Skyler Likhomanenko, Tatiana Shum, Stephen Abdelaziz, Ahmed Hussen Watanabe, Shinji Theobald, Barry-John Sound Audio and Speech Processing Self-supervised features are typically used in place of filter-bank features in speaker verification models. However, these models were originally designed to ingest filter-bank features as inputs, and thus, training them on top of self-supervised features assumes that both feature types require the same amount of learning for the task. In this work, we observe that pre-trained self-supervised speech features inherently include information required for downstream speaker verification task, and therefore, we can simplify the downstream model without sacrificing performance. To this end, we revisit the design of the downstream model for speaker verification using self-supervised features. We show that we can simplify the model to use 97.51% fewer parameters while achieving a 29.93% average improvement in performance on SUPERB. Consequently, we show that the simplified downstream model is more data efficient compared to baseline--it achieves better performance with only 60% of the training data. |
| title | Can you Remove the Downstream Model for Speaker Recognition with Self-Supervised Speech Features? |
| topic | Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2402.00340 |