Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2408.04637 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913462931685376 |
|---|---|
| author | Qian, Kun Sang, Yisi Bayat, Farima Fatahi Belyi, Anton Chu, Xianqi Govind, Yash Khorshidi, Samira Khot, Rahul Luna, Katherine Nikfarjam, Azadeh Qi, Xiaoguang Wu, Fei Zhang, Xianhan Li, Yunyao |
| author_facet | Qian, Kun Sang, Yisi Bayat, Farima Fatahi Belyi, Anton Chu, Xianqi Govind, Yash Khorshidi, Samira Khot, Rahul Luna, Katherine Nikfarjam, Azadeh Qi, Xiaoguang Wu, Fei Zhang, Xianhan Li, Yunyao |
| contents | Prompt engineering is an iterative procedure often requiring extensive manual effort to formulate suitable instructions for effectively directing large language models (LLMs) in specific tasks. Incorporating few-shot examples is a vital and effective approach to providing LLMs with precise instructions, leading to improved LLM performance. Nonetheless, identifying the most informative demonstrations for LLMs is labor-intensive, frequently entailing sifting through an extensive search space. In this demonstration, we showcase a human-in-the-loop tool called APE (Active Prompt Engineering) designed for refining prompts through active learning. Drawing inspiration from active learning, APE iteratively selects the most ambiguous examples for human feedback, which will be transformed into few-shot examples within the prompt. The demo recording can be found with the submission or be viewed at https://youtu.be/OwQ6MQx53-Y. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_04637 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | APE: Active Learning-based Tooling for Finding Informative Few-shot Examples for LLM-based Entity Matching Qian, Kun Sang, Yisi Bayat, Farima Fatahi Belyi, Anton Chu, Xianqi Govind, Yash Khorshidi, Samira Khot, Rahul Luna, Katherine Nikfarjam, Azadeh Qi, Xiaoguang Wu, Fei Zhang, Xianhan Li, Yunyao Computation and Language Prompt engineering is an iterative procedure often requiring extensive manual effort to formulate suitable instructions for effectively directing large language models (LLMs) in specific tasks. Incorporating few-shot examples is a vital and effective approach to providing LLMs with precise instructions, leading to improved LLM performance. Nonetheless, identifying the most informative demonstrations for LLMs is labor-intensive, frequently entailing sifting through an extensive search space. In this demonstration, we showcase a human-in-the-loop tool called APE (Active Prompt Engineering) designed for refining prompts through active learning. Drawing inspiration from active learning, APE iteratively selects the most ambiguous examples for human feedback, which will be transformed into few-shot examples within the prompt. The demo recording can be found with the submission or be viewed at https://youtu.be/OwQ6MQx53-Y. |
| title | APE: Active Learning-based Tooling for Finding Informative Few-shot Examples for LLM-based Entity Matching |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2408.04637 |