Prompt Selection Matters: Enhancing Text Annotations for Social Sciences with Large Language Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913726176690176 |
|---|---|
| author | Abraham, Louis Arnal, Charles Marie, Antoine |
| author_facet | Abraham, Louis Arnal, Charles Marie, Antoine |
| contents | Large Language Models have recently been applied to text annotation tasks from social sciences, equalling or surpassing the performance of human workers at a fraction of the cost. However, no inquiry has yet been made on the impact of prompt selection on labelling accuracy. In this study, we show that performance greatly varies between prompts, and we apply the method of automatic prompt optimization to systematically craft high quality prompts. We also provide the community with a simple, browser-based implementation of the method at https://prompt-ultra.github.io/ . |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_10645 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Prompt Selection Matters: Enhancing Text Annotations for Social Sciences with Large Language Models Abraham, Louis Arnal, Charles Marie, Antoine Computation and Language Artificial Intelligence Computers and Society Large Language Models have recently been applied to text annotation tasks from social sciences, equalling or surpassing the performance of human workers at a fraction of the cost. However, no inquiry has yet been made on the impact of prompt selection on labelling accuracy. In this study, we show that performance greatly varies between prompts, and we apply the method of automatic prompt optimization to systematically craft high quality prompts. We also provide the community with a simple, browser-based implementation of the method at https://prompt-ultra.github.io/ . |
| title | Prompt Selection Matters: Enhancing Text Annotations for Social Sciences with Large Language Models |
| topic | Computation and Language Artificial Intelligence Computers and Society |
| url | https://arxiv.org/abs/2407.10645 |