Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2311.09782 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911822603354112 |
|---|---|
| author | Yao, Bingsheng Chen, Guiming Zou, Ruishi Lu, Yuxuan Li, Jiachen Zhang, Shao Sang, Yisi Liu, Sijia Hendler, James Wang, Dakuo |
| author_facet | Yao, Bingsheng Chen, Guiming Zou, Ruishi Lu, Yuxuan Li, Jiachen Zhang, Shao Sang, Yisi Liu, Sijia Hendler, James Wang, Dakuo |
| contents | While most existing works on LLM prompting techniques focus only on how to select a better set of data samples inside one single prompt input (In-Context Learning or ICL), why can not we design and leverage multiple prompts together to further improve the LLM's performance? In this work, we propose In-Context Sampling (ICS), a low-resource LLM prompting technique to produce confident predictions by optimizing the construction of multiple ICL prompt inputs. Extensive experiments with three open-source LLMs (FlanT5-XL, Mistral-7B, and Mixtral-8x7B) on four NLI datasets (e-SNLI, Multi-NLI, ANLI, and Contract-NLI) and one QA dataset (CommonsenseQA) illustrate that ICS can consistently enhance LLMs' performance. An in-depth evaluation with three data similarity-based ICS strategies suggests that these strategies can further elevate LLM's performance, which sheds light on a new yet promising future research direction. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_09782 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | More Samples or More Prompts? Exploring Effective In-Context Sampling for LLM Few-Shot Prompt Engineering Yao, Bingsheng Chen, Guiming Zou, Ruishi Lu, Yuxuan Li, Jiachen Zhang, Shao Sang, Yisi Liu, Sijia Hendler, James Wang, Dakuo Computation and Language While most existing works on LLM prompting techniques focus only on how to select a better set of data samples inside one single prompt input (In-Context Learning or ICL), why can not we design and leverage multiple prompts together to further improve the LLM's performance? In this work, we propose In-Context Sampling (ICS), a low-resource LLM prompting technique to produce confident predictions by optimizing the construction of multiple ICL prompt inputs. Extensive experiments with three open-source LLMs (FlanT5-XL, Mistral-7B, and Mixtral-8x7B) on four NLI datasets (e-SNLI, Multi-NLI, ANLI, and Contract-NLI) and one QA dataset (CommonsenseQA) illustrate that ICS can consistently enhance LLMs' performance. An in-depth evaluation with three data similarity-based ICS strategies suggests that these strategies can further elevate LLM's performance, which sheds light on a new yet promising future research direction. |
| title | More Samples or More Prompts? Exploring Effective In-Context Sampling for LLM Few-Shot Prompt Engineering |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2311.09782 |