Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2022
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2210.17437 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914953939648512 |
|---|---|
| author | Singh, Avyav Kumar Shutova, Ekaterina Yannakoudakis, Helen |
| author_facet | Singh, Avyav Kumar Shutova, Ekaterina Yannakoudakis, Helen |
| contents | Existing approaches to few-shot learning in NLP rely on large language models (LLMs) and/or fine-tuning of these to generalise on out-of-distribution data. In this work, we propose a novel few-shot learning approach based on soft-label prototypes (SLPs) designed to collectively capture the distribution of different classes across the input domain space. We focus on learning previously unseen NLP tasks from very few examples (4, 8, 16) per class and experimentally demonstrate that our approach achieves superior performance on the majority of tested tasks in this data-lean setting while being highly parameter efficient. We also show that our few-shot adaptation method can be integrated into more generalised learning settings, primarily meta-learning, to yield superior performance against strong baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2210_17437 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Learning New Tasks from a Few Examples with Soft-Label Prototypes Singh, Avyav Kumar Shutova, Ekaterina Yannakoudakis, Helen Machine Learning Computation and Language Existing approaches to few-shot learning in NLP rely on large language models (LLMs) and/or fine-tuning of these to generalise on out-of-distribution data. In this work, we propose a novel few-shot learning approach based on soft-label prototypes (SLPs) designed to collectively capture the distribution of different classes across the input domain space. We focus on learning previously unseen NLP tasks from very few examples (4, 8, 16) per class and experimentally demonstrate that our approach achieves superior performance on the majority of tested tasks in this data-lean setting while being highly parameter efficient. We also show that our few-shot adaptation method can be integrated into more generalised learning settings, primarily meta-learning, to yield superior performance against strong baselines. |
| title | Learning New Tasks from a Few Examples with Soft-Label Prototypes |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2210.17437 |