MT-SLVR: Multi-Task Self-Supervised Learning for Transformation In(Variant) Representations
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909083262517248 |
|---|---|
| author | Heggan, Calum Hospedales, Tim Budgett, Sam Yaghoobi, Mehrdad |
| author_facet | Heggan, Calum Hospedales, Tim Budgett, Sam Yaghoobi, Mehrdad |
| contents | Contrastive self-supervised learning has gained attention for its ability to create high-quality representations from large unlabelled data sets. A key reason that these powerful features enable data-efficient learning of downstream tasks is that they provide augmentation invariance, which is often a useful inductive bias. However, the amount and type of invariances preferred is not known apriori, and varies across different downstream tasks. We therefore propose a multi-task self-supervised framework (MT-SLVR) that learns both variant and invariant features in a parameter-efficient manner. Our multi-task representation provides a strong and flexible feature that benefits diverse downstream tasks. We evaluate our approach on few-shot classification tasks drawn from a variety of audio domains and demonstrate improved classification performance on all of them |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_17191 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | MT-SLVR: Multi-Task Self-Supervised Learning for Transformation In(Variant) Representations Heggan, Calum Hospedales, Tim Budgett, Sam Yaghoobi, Mehrdad Machine Learning Sound Audio and Speech Processing Contrastive self-supervised learning has gained attention for its ability to create high-quality representations from large unlabelled data sets. A key reason that these powerful features enable data-efficient learning of downstream tasks is that they provide augmentation invariance, which is often a useful inductive bias. However, the amount and type of invariances preferred is not known apriori, and varies across different downstream tasks. We therefore propose a multi-task self-supervised framework (MT-SLVR) that learns both variant and invariant features in a parameter-efficient manner. Our multi-task representation provides a strong and flexible feature that benefits diverse downstream tasks. We evaluate our approach on few-shot classification tasks drawn from a variety of audio domains and demonstrate improved classification performance on all of them |
| title | MT-SLVR: Multi-Task Self-Supervised Learning for Transformation In(Variant) Representations |
| topic | Machine Learning Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2305.17191 |