Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2022
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2211.04118 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913261294714880 |
|---|---|
| author | Weng, Jinta Deng, Yifan Li, d Donghao You, Hao Hu, Yue Huang, Heyan |
| author_facet | Weng, Jinta Deng, Yifan Li, d Donghao You, Hao Hu, Yue Huang, Heyan |
| contents | The prompt has become an effective linguistic tool for utilizing pre-trained language models. However, in few-shot scenarios, subtle changes in the prompt design always make the result widely different, and the prompt learning methods also make it easy to overfit the limited samples. To alleviate this, we explore utilizing suitable contrastive samples and multi-degree contrastive learning methods to improve the robustness of the prompt representation. Therefore, the proposed Consprompt combined with the prompt encoding network, contrastive sampling modules, and contrastive scoring modules, is introduced to realize differential contrastive learning. Our results exhibit state-of-the-art performance in different few-shot settings, and the ablation experiments also certify the effectiveness of utilizing multi-degree contrastive learning in the prompt-based fine-tuning process. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2211_04118 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | ConsPrompt: Exploiting Contrastive Samples for Fewshot Prompt Learning Weng, Jinta Deng, Yifan Li, d Donghao You, Hao Hu, Yue Huang, Heyan Computation and Language Artificial Intelligence The prompt has become an effective linguistic tool for utilizing pre-trained language models. However, in few-shot scenarios, subtle changes in the prompt design always make the result widely different, and the prompt learning methods also make it easy to overfit the limited samples. To alleviate this, we explore utilizing suitable contrastive samples and multi-degree contrastive learning methods to improve the robustness of the prompt representation. Therefore, the proposed Consprompt combined with the prompt encoding network, contrastive sampling modules, and contrastive scoring modules, is introduced to realize differential contrastive learning. Our results exhibit state-of-the-art performance in different few-shot settings, and the ablation experiments also certify the effectiveness of utilizing multi-degree contrastive learning in the prompt-based fine-tuning process. |
| title | ConsPrompt: Exploiting Contrastive Samples for Fewshot Prompt Learning |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2211.04118 |