A Survey on In-context Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2022
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916423745404928 |
|---|---|
| author | Dong, Qingxiu Li, Lei Dai, Damai Zheng, Ce Ma, Jingyuan Li, Rui Xia, Heming Xu, Jingjing Wu, Zhiyong Liu, Tianyu Chang, Baobao Sun, Xu Li, Lei Sui, Zhifang |
| author_facet | Dong, Qingxiu Li, Lei Dai, Damai Zheng, Ce Ma, Jingyuan Li, Rui Xia, Heming Xu, Jingjing Wu, Zhiyong Liu, Tianyu Chang, Baobao Sun, Xu Li, Lei Sui, Zhifang |
| contents | With the increasing capabilities of large language models (LLMs), in-context learning (ICL) has emerged as a new paradigm for natural language processing (NLP), where LLMs make predictions based on contexts augmented with a few examples. It has been a significant trend to explore ICL to evaluate and extrapolate the ability of LLMs. In this paper, we aim to survey and summarize the progress and challenges of ICL. We first present a formal definition of ICL and clarify its correlation to related studies. Then, we organize and discuss advanced techniques, including training strategies, prompt designing strategies, and related analysis. Additionally, we explore various ICL application scenarios, such as data engineering and knowledge updating. Finally, we address the challenges of ICL and suggest potential directions for further research. We hope that our work can encourage more research on uncovering how ICL works and improving ICL. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2301_00234 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | A Survey on In-context Learning Dong, Qingxiu Li, Lei Dai, Damai Zheng, Ce Ma, Jingyuan Li, Rui Xia, Heming Xu, Jingjing Wu, Zhiyong Liu, Tianyu Chang, Baobao Sun, Xu Li, Lei Sui, Zhifang Computation and Language Artificial Intelligence With the increasing capabilities of large language models (LLMs), in-context learning (ICL) has emerged as a new paradigm for natural language processing (NLP), where LLMs make predictions based on contexts augmented with a few examples. It has been a significant trend to explore ICL to evaluate and extrapolate the ability of LLMs. In this paper, we aim to survey and summarize the progress and challenges of ICL. We first present a formal definition of ICL and clarify its correlation to related studies. Then, we organize and discuss advanced techniques, including training strategies, prompt designing strategies, and related analysis. Additionally, we explore various ICL application scenarios, such as data engineering and knowledge updating. Finally, we address the challenges of ICL and suggest potential directions for further research. We hope that our work can encourage more research on uncovering how ICL works and improving ICL. |
| title | A Survey on In-context Learning |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2301.00234 |