A Technological Construction of Society: comparing GPT-4 and human respondents for occupational evaluation in the UK
Fuente:
Organización Internacional del Trabajo (OIT)
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Entrada de enciclopedia |
| Veröffentlicht: |
International Labour Organization
2024
|
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1874524980472971265 |
|---|---|
| author | Pawel Gmyrek (Author) Christoph Lutz (Author) Gemma Newlands (Author) |
| author_facet | Pawel Gmyrek (Author) Christoph Lutz (Author) Gemma Newlands (Author) Pawel Gmyrek (Author) Christoph Lutz (Author) Gemma Newlands (Author) |
| contents | A Technological Construction of Society: comparing GPT-4 and human respondents for occupational evaluation in the UK Pawel Gmyrek (Author) Christoph Lutz (Author) Gemma Newlands (Author) Despite initial research about the biases and perceptions of Large Language Models (LLMs), we lack evidence on how LLMs evaluate occupations, especially in comparison to human evaluators. In this paper, we present a systematic comparison of occupational evaluations by GPT-4 with those from an in-depth, high-quality and recent human respondents survey in the United Kingdom. Covering the full ISCO-08 occupational landscape, with 580 occupations and two distinct metrics (prestige and social value), our findings indicate that GPT-4 and human scores are highly correlated across all ISCO-08 major groups. In absolute terms, GPT-4 scores are more generous than those of the human respondents. At the same time, GPT-4 substantially under or overestimates the occupational prestige and social value of many occupations, particularly for emerging digital and stigmatized occupations. ILO working paper, 102, ILO 10.54394/UQOQ5153 DOI https://doi.org/10.54394/UQOQ5153; ISBN 9789220403266 publication.encyclopediaEntry |
| format | Entrada de enciclopedia |
| id | ilo_995352793502676 |
| institution | Organización Internacional del Trabajo (OIT) |
| publishDate | 2024 |
| publisher | International Labour Organization |
| spellingShingle | A Technological Construction of Society: comparing GPT-4 and human respondents for occupational evaluation in the UK Pawel Gmyrek (Author) Christoph Lutz (Author) Gemma Newlands (Author) A Technological Construction of Society: comparing GPT-4 and human respondents for occupational evaluation in the UK Pawel Gmyrek (Author) Christoph Lutz (Author) Gemma Newlands (Author) Despite initial research about the biases and perceptions of Large Language Models (LLMs), we lack evidence on how LLMs evaluate occupations, especially in comparison to human evaluators. In this paper, we present a systematic comparison of occupational evaluations by GPT-4 with those from an in-depth, high-quality and recent human respondents survey in the United Kingdom. Covering the full ISCO-08 occupational landscape, with 580 occupations and two distinct metrics (prestige and social value), our findings indicate that GPT-4 and human scores are highly correlated across all ISCO-08 major groups. In absolute terms, GPT-4 scores are more generous than those of the human respondents. At the same time, GPT-4 substantially under or overestimates the occupational prestige and social value of many occupations, particularly for emerging digital and stigmatized occupations. ILO working paper, 102, ILO 10.54394/UQOQ5153 DOI https://doi.org/10.54394/UQOQ5153; ISBN 9789220403266 publication.encyclopediaEntry |
| title | A Technological Construction of Society: comparing GPT-4 and human respondents for occupational evaluation in the UK |
| url | https://researchrepository.ilo.org/esploro/outputs/encyclopediaEntry/A-Technological-Construction-of-Society-comparing/995352793502676 |