Personas with Attitudes: Controlling LLMs for Diverse Data Annotation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917804045762560 |
|---|---|
| author | Fröhling, Leon Demartini, Gianluca Assenmacher, Dennis |
| author_facet | Fröhling, Leon Demartini, Gianluca Assenmacher, Dennis |
| contents | We present a novel approach for enhancing diversity and control in data annotation tasks by personalizing large language models (LLMs). We investigate the impact of injecting diverse persona descriptions into LLM prompts across two studies, exploring whether personas increase annotation diversity and whether the impacts of individual personas on the resulting annotations are consistent and controllable. Our results show that persona-prompted LLMs produce more diverse annotations than LLMs prompted without personas and that these effects are both controllable and repeatable, making our approach a suitable tool for improving data annotation in subjective NLP tasks like toxicity detection. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_11745 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Personas with Attitudes: Controlling LLMs for Diverse Data Annotation Fröhling, Leon Demartini, Gianluca Assenmacher, Dennis Computation and Language Human-Computer Interaction We present a novel approach for enhancing diversity and control in data annotation tasks by personalizing large language models (LLMs). We investigate the impact of injecting diverse persona descriptions into LLM prompts across two studies, exploring whether personas increase annotation diversity and whether the impacts of individual personas on the resulting annotations are consistent and controllable. Our results show that persona-prompted LLMs produce more diverse annotations than LLMs prompted without personas and that these effects are both controllable and repeatable, making our approach a suitable tool for improving data annotation in subjective NLP tasks like toxicity detection. |
| title | Personas with Attitudes: Controlling LLMs for Diverse Data Annotation |
| topic | Computation and Language Human-Computer Interaction |
| url | https://arxiv.org/abs/2410.11745 |