Manual Verbalizer Enrichment for Few-Shot Text Classification
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912071257423872 |
|---|---|
| author | Nguyen, Quang Anh Tomeh, Nadi Lebbah, Mustapha Charnois, Thierry Azzag, Hanene Muñoz, Santiago Cordoba |
| author_facet | Nguyen, Quang Anh Tomeh, Nadi Lebbah, Mustapha Charnois, Thierry Azzag, Hanene Muñoz, Santiago Cordoba |
| contents | With the continuous development of pre-trained language models, prompt-based training becomes a well-adopted paradigm that drastically improves the exploitation of models for many natural language processing tasks. Prompting also shows great performance compared to traditional fine-tuning when adapted to zero-shot or few-shot scenarios where the number of annotated data is limited. In this framework, the role of verbalizers is essential, as an interpretation from masked word distributions into output predictions. In this work, we propose \acrshort{mave}, an approach for verbalizer construction by enrichment of class labels using neighborhood relation in the embedding space of words for the text classification task. In addition, we elaborate a benchmarking procedure to evaluate typical baselines of verbalizers for document classification in few-shot learning contexts. Our model achieves state-of-the-art results while using significantly fewer resources. We show that our approach is particularly effective in cases with extremely limited supervision data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_06173 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Manual Verbalizer Enrichment for Few-Shot Text Classification Nguyen, Quang Anh Tomeh, Nadi Lebbah, Mustapha Charnois, Thierry Azzag, Hanene Muñoz, Santiago Cordoba Computation and Language Artificial Intelligence Machine Learning With the continuous development of pre-trained language models, prompt-based training becomes a well-adopted paradigm that drastically improves the exploitation of models for many natural language processing tasks. Prompting also shows great performance compared to traditional fine-tuning when adapted to zero-shot or few-shot scenarios where the number of annotated data is limited. In this framework, the role of verbalizers is essential, as an interpretation from masked word distributions into output predictions. In this work, we propose \acrshort{mave}, an approach for verbalizer construction by enrichment of class labels using neighborhood relation in the embedding space of words for the text classification task. In addition, we elaborate a benchmarking procedure to evaluate typical baselines of verbalizers for document classification in few-shot learning contexts. Our model achieves state-of-the-art results while using significantly fewer resources. We show that our approach is particularly effective in cases with extremely limited supervision data. |
| title | Manual Verbalizer Enrichment for Few-Shot Text Classification |
| topic | Computation and Language Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2410.06173 |