Effective In-Context Example Selection through Data Compression
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929348607475712 |
|---|---|
| author | Sun, Zhongxiang Zhang, Kepu Wang, Haoyu Zhang, Xiao Xu, Jun |
| author_facet | Sun, Zhongxiang Zhang, Kepu Wang, Haoyu Zhang, Xiao Xu, Jun |
| contents | In-context learning has been extensively validated in large language models. However, the mechanism and selection strategy for in-context example selection, which is a crucial ingredient in this approach, lacks systematic and in-depth research. In this paper, we propose a data compression approach to the selection of in-context examples. We introduce a two-stage method that can effectively choose relevant examples and retain sufficient information about the training dataset within the in-context examples. Our method shows a significant improvement of an average of 5.90% across five different real-world datasets using four language models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_11465 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Effective In-Context Example Selection through Data Compression Sun, Zhongxiang Zhang, Kepu Wang, Haoyu Zhang, Xiao Xu, Jun Computation and Language In-context learning has been extensively validated in large language models. However, the mechanism and selection strategy for in-context example selection, which is a crucial ingredient in this approach, lacks systematic and in-depth research. In this paper, we propose a data compression approach to the selection of in-context examples. We introduce a two-stage method that can effectively choose relevant examples and retain sufficient information about the training dataset within the in-context examples. Our method shows a significant improvement of an average of 5.90% across five different real-world datasets using four language models. |
| title | Effective In-Context Example Selection through Data Compression |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2405.11465 |