Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2401.12295 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915411562332160 |
|---|---|
| author | Castro-Gonzalez, Leonardo Chung, Yi-Ling Kirk, Hannak Rose Francis, John Williams, Angus R. Johansson, Pica Bright, Jonathan |
| author_facet | Castro-Gonzalez, Leonardo Chung, Yi-Ling Kirk, Hannak Rose Francis, John Williams, Angus R. Johansson, Pica Bright, Jonathan |
| contents | The field of machine learning has recently made significant progress in reducing the requirements for labelled training data when building new models. These `cheaper' learning techniques hold significant potential for the social sciences, where development of large labelled training datasets is often a significant practical impediment to the use of machine learning for analytical tasks. In this article we review three `cheap' techniques that have developed in recent years: weak supervision, transfer learning and prompt engineering. For the latter, we also review the particular case of zero-shot prompting of large language models. For each technique we provide a guide of how it works and demonstrate its application across six different realistic social science applications (two different tasks paired with three different dataset makeups). We show good performance for all techniques, and in particular we demonstrate how prompting of large language models can achieve high accuracy at very low cost. Our results are accompanied by a code repository to make it easy for others to duplicate our work and use it in their own research. Overall, our article is intended to stimulate further uptake of these techniques in the social sciences. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_12295 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Cheap Learning: Maximising Performance of Language Models for Social Data Science Using Minimal Data Castro-Gonzalez, Leonardo Chung, Yi-Ling Kirk, Hannak Rose Francis, John Williams, Angus R. Johansson, Pica Bright, Jonathan Computation and Language I.2.7; J.4 The field of machine learning has recently made significant progress in reducing the requirements for labelled training data when building new models. These `cheaper' learning techniques hold significant potential for the social sciences, where development of large labelled training datasets is often a significant practical impediment to the use of machine learning for analytical tasks. In this article we review three `cheap' techniques that have developed in recent years: weak supervision, transfer learning and prompt engineering. For the latter, we also review the particular case of zero-shot prompting of large language models. For each technique we provide a guide of how it works and demonstrate its application across six different realistic social science applications (two different tasks paired with three different dataset makeups). We show good performance for all techniques, and in particular we demonstrate how prompting of large language models can achieve high accuracy at very low cost. Our results are accompanied by a code repository to make it easy for others to duplicate our work and use it in their own research. Overall, our article is intended to stimulate further uptake of these techniques in the social sciences. |
| title | Cheap Learning: Maximising Performance of Language Models for Social Data Science Using Minimal Data |
| topic | Computation and Language I.2.7; J.4 |
| url | https://arxiv.org/abs/2401.12295 |