MT-SLVR: Multi-Task Self-Supervised Learning for Transformation In(Variant) Representations
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909083262517248 |
|---|---|
| author | Heggan, Calum Hospedales, Tim Budgett, Sam Yaghoobi, Mehrdad |
| author_facet | Heggan, Calum Hospedales, Tim Budgett, Sam Yaghoobi, Mehrdad |
| contents | Contrastive self-supervised learning has gained attention for its ability to create high-quality representations from large unlabelled data sets. A key reason that these powerful features enable data-efficient learning of downstream tasks is that they provide augmentation invariance, which is often a useful inductive bias. However, the amount and type of invariances preferred is not known apriori, and varies across different downstream tasks. We therefore propose a multi-task self-supervised framework (MT-SLVR) that learns both variant and invariant features in a parameter-efficient manner. Our multi-task representation provides a strong and flexible feature that benefits diverse downstream tasks. We evaluate our approach on few-shot classification tasks drawn from a variety of audio domains and demonstrate improved classification performance on all of them |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_17191 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | MT-SLVR: Multi-Task Self-Supervised Learning for Transformation In(Variant) Representations Heggan, Calum Hospedales, Tim Budgett, Sam Yaghoobi, Mehrdad Machine Learning Sound Audio and Speech Processing Contrastive self-supervised learning has gained attention for its ability to create high-quality representations from large unlabelled data sets. A key reason that these powerful features enable data-efficient learning of downstream tasks is that they provide augmentation invariance, which is often a useful inductive bias. However, the amount and type of invariances preferred is not known apriori, and varies across different downstream tasks. We therefore propose a multi-task self-supervised framework (MT-SLVR) that learns both variant and invariant features in a parameter-efficient manner. Our multi-task representation provides a strong and flexible feature that benefits diverse downstream tasks. We evaluate our approach on few-shot classification tasks drawn from a variety of audio domains and demonstrate improved classification performance on all of them |
| title | MT-SLVR: Multi-Task Self-Supervised Learning for Transformation In(Variant) Representations |
| topic | Machine Learning Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2305.17191 |