ExpertPrompting: Instructing Large Language Models to be Distinguished Experts
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908255908790272 |
|---|---|
| author | Xu, Benfeng Yang, An Lin, Junyang Wang, Quan Zhou, Chang Zhang, Yongdong Mao, Zhendong |
| author_facet | Xu, Benfeng Yang, An Lin, Junyang Wang, Quan Zhou, Chang Zhang, Yongdong Mao, Zhendong |
| contents | The answering quality of an aligned large language model (LLM) can be drastically improved if treated with proper crafting of prompts. In this paper, we propose ExpertPrompting to elicit the potential of LLMs to answer as distinguished experts. We first utilize In-Context Learning to automatically synthesize detailed and customized descriptions of the expert identity for each specific instruction, and then ask LLMs to provide answer conditioned on such agent background. Based on this augmented prompting strategy, we produce a new set of instruction-following data using GPT-3.5, and train a competitive open-source chat assistant called ExpertLLaMA. We employ GPT4-based evaluation to show that 1) the expert data is of significantly higher quality than vanilla answers, and 2) ExpertLLaMA outperforms existing open-source opponents and achieves 96\% of the original ChatGPT's capability. All data and the ExpertLLaMA model will be made publicly available at https://github.com/OFA-Sys/ExpertLLaMA. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_14688 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | ExpertPrompting: Instructing Large Language Models to be Distinguished Experts Xu, Benfeng Yang, An Lin, Junyang Wang, Quan Zhou, Chang Zhang, Yongdong Mao, Zhendong Computation and Language Artificial Intelligence The answering quality of an aligned large language model (LLM) can be drastically improved if treated with proper crafting of prompts. In this paper, we propose ExpertPrompting to elicit the potential of LLMs to answer as distinguished experts. We first utilize In-Context Learning to automatically synthesize detailed and customized descriptions of the expert identity for each specific instruction, and then ask LLMs to provide answer conditioned on such agent background. Based on this augmented prompting strategy, we produce a new set of instruction-following data using GPT-3.5, and train a competitive open-source chat assistant called ExpertLLaMA. We employ GPT4-based evaluation to show that 1) the expert data is of significantly higher quality than vanilla answers, and 2) ExpertLLaMA outperforms existing open-source opponents and achieves 96\% of the original ChatGPT's capability. All data and the ExpertLLaMA model will be made publicly available at https://github.com/OFA-Sys/ExpertLLaMA. |
| title | ExpertPrompting: Instructing Large Language Models to be Distinguished Experts |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2305.14688 |