Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2401.08420 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909074961989632 |
|---|---|
| author | Balloccu, Simone Reiter, Ehud Kumar, Vivek Recupero, Diego Reforgiato Riboni, Daniele |
| author_facet | Balloccu, Simone Reiter, Ehud Kumar, Vivek Recupero, Diego Reforgiato Riboni, Daniele |
| contents | Large Language Models (LLMs), with their flexible generation abilities, can be powerful data sources in domains with few or no available corpora. However, problems like hallucinations and biases limit such applications. In this case study, we pick nutrition counselling, a domain lacking any public resource, and show that high-quality datasets can be gathered by combining LLMs, crowd-workers and nutrition experts. We first crowd-source and cluster a novel dataset of diet-related issues, then work with experts to prompt ChatGPT into producing related supportive text. Finally, we let the experts evaluate the safety of the generated text. We release HAI-coaching, the first expert-annotated nutrition counselling dataset containing ~2.4K dietary struggles from crowd workers, and ~97K related supportive texts generated by ChatGPT. Extensive analysis shows that ChatGPT while producing highly fluent and human-like text, also manifests harmful behaviours, especially in sensitive topics like mental health, making it unsuitable for unsupervised use. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_08420 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Ask the experts: sourcing high-quality datasets for nutritional counselling through Human-AI collaboration Balloccu, Simone Reiter, Ehud Kumar, Vivek Recupero, Diego Reforgiato Riboni, Daniele Computation and Language Large Language Models (LLMs), with their flexible generation abilities, can be powerful data sources in domains with few or no available corpora. However, problems like hallucinations and biases limit such applications. In this case study, we pick nutrition counselling, a domain lacking any public resource, and show that high-quality datasets can be gathered by combining LLMs, crowd-workers and nutrition experts. We first crowd-source and cluster a novel dataset of diet-related issues, then work with experts to prompt ChatGPT into producing related supportive text. Finally, we let the experts evaluate the safety of the generated text. We release HAI-coaching, the first expert-annotated nutrition counselling dataset containing ~2.4K dietary struggles from crowd workers, and ~97K related supportive texts generated by ChatGPT. Extensive analysis shows that ChatGPT while producing highly fluent and human-like text, also manifests harmful behaviours, especially in sensitive topics like mental health, making it unsuitable for unsupervised use. |
| title | Ask the experts: sourcing high-quality datasets for nutritional counselling through Human-AI collaboration |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2401.08420 |