Curriculum Demonstration Selection for In-Context Learning
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917869796720640 |
|---|---|
| author | Vu, Duc Anh Duy, Nguyen Tran Cong Wu, Xiaobao Nhat, Hoang Minh Mingzhe, Du Thong, Nguyen Thanh Luu, Anh Tuan |
| author_facet | Vu, Duc Anh Duy, Nguyen Tran Cong Wu, Xiaobao Nhat, Hoang Minh Mingzhe, Du Thong, Nguyen Thanh Luu, Anh Tuan |
| contents | Large Language Models (LLMs) have shown strong in-context learning (ICL) abilities with a few demonstrations. However, one critical challenge is how to select demonstrations to elicit the full potential of LLMs. In this paper, we propose Curriculum Demonstration Selection (CDS), a novel demonstration selection method for ICL. Instead of merely using similarity, CDS additionally partitions samples by their complexity measurements. Following curriculum learning, CDS then selects demonstrations from easy to difficult. Thus the selected demonstrations cover a wide range of difficulty levels, enabling LLMs to learn from varied complexities within the training set. Experiments demonstrate that our CDS consistently outperforms baseline methods, achieving notable improvements across nine LLMs on three benchmarks. Moreover, CDS proves especially effective in enhancing LLM performance in solving challenging problems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_18126 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Curriculum Demonstration Selection for In-Context Learning Vu, Duc Anh Duy, Nguyen Tran Cong Wu, Xiaobao Nhat, Hoang Minh Mingzhe, Du Thong, Nguyen Thanh Luu, Anh Tuan Computation and Language Large Language Models (LLMs) have shown strong in-context learning (ICL) abilities with a few demonstrations. However, one critical challenge is how to select demonstrations to elicit the full potential of LLMs. In this paper, we propose Curriculum Demonstration Selection (CDS), a novel demonstration selection method for ICL. Instead of merely using similarity, CDS additionally partitions samples by their complexity measurements. Following curriculum learning, CDS then selects demonstrations from easy to difficult. Thus the selected demonstrations cover a wide range of difficulty levels, enabling LLMs to learn from varied complexities within the training set. Experiments demonstrate that our CDS consistently outperforms baseline methods, achieving notable improvements across nine LLMs on three benchmarks. Moreover, CDS proves especially effective in enhancing LLM performance in solving challenging problems. |
| title | Curriculum Demonstration Selection for In-Context Learning |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2411.18126 |