Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2401.09783 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917570124185600 |
|---|---|
| author | Zhang, Yong Li, Hanzhang Li, Zhitao Cheng, Ning Li, Ming Xiao, Jing Wang, Jianzong |
| author_facet | Zhang, Yong Li, Hanzhang Li, Zhitao Cheng, Ning Li, Ming Xiao, Jing Wang, Jianzong |
| contents | Large Language Models (LLMs) have shown significant promise in various applications, including zero-shot and few-shot learning. However, their performance can be hampered by inherent biases. Instead of traditionally sought methods that aim to minimize or correct these biases, this study introduces a novel methodology named ``bias-kNN''. This approach capitalizes on the biased outputs, harnessing them as primary features for kNN and supplementing with gold labels. Our comprehensive evaluations, spanning diverse domain text classification datasets and different GPT-2 model sizes, indicate the adaptability and efficacy of the ``bias-kNN'' method. Remarkably, this approach not only outperforms conventional in-context learning in few-shot scenarios but also demonstrates robustness across a spectrum of samples, templates and verbalizers. This study, therefore, presents a unique perspective on harnessing biases, transforming them into assets for enhanced model performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_09783 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Leveraging Biases in Large Language Models: "bias-kNN'' for Effective Few-Shot Learning Zhang, Yong Li, Hanzhang Li, Zhitao Cheng, Ning Li, Ming Xiao, Jing Wang, Jianzong Computation and Language Large Language Models (LLMs) have shown significant promise in various applications, including zero-shot and few-shot learning. However, their performance can be hampered by inherent biases. Instead of traditionally sought methods that aim to minimize or correct these biases, this study introduces a novel methodology named ``bias-kNN''. This approach capitalizes on the biased outputs, harnessing them as primary features for kNN and supplementing with gold labels. Our comprehensive evaluations, spanning diverse domain text classification datasets and different GPT-2 model sizes, indicate the adaptability and efficacy of the ``bias-kNN'' method. Remarkably, this approach not only outperforms conventional in-context learning in few-shot scenarios but also demonstrates robustness across a spectrum of samples, templates and verbalizers. This study, therefore, presents a unique perspective on harnessing biases, transforming them into assets for enhanced model performance. |
| title | Leveraging Biases in Large Language Models: "bias-kNN'' for Effective Few-Shot Learning |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2401.09783 |