Troubling Taxonomies in GenAI Evaluation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909372348628992 |
|---|---|
| author | Berman, Glen Cooper, Ned Deng, Wesley Hanwen Hutchinson, Ben |
| author_facet | Berman, Glen Cooper, Ned Deng, Wesley Hanwen Hutchinson, Ben |
| contents | To evaluate the societal impacts of GenAI requires a model of how social harms emerge from interactions between GenAI, people, and societal structures. Yet a model is rarely explicitly defined in societal impact evaluations, or in the taxonomies of societal impacts that support them. In this provocation, we argue that societal impacts should be conceptualised as application- and context-specific, incommensurable, and shaped by questions of social power. Doing so leads us to conclude that societal impact evaluations using existing taxonomies are inherently limited, in terms of their potential to reveal how GenAI systems may interact with people when introduced into specific social contexts. We therefore propose a governance-first approach to managing societal harms attended by GenAI technologies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_22985 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Troubling Taxonomies in GenAI Evaluation Berman, Glen Cooper, Ned Deng, Wesley Hanwen Hutchinson, Ben Human-Computer Interaction To evaluate the societal impacts of GenAI requires a model of how social harms emerge from interactions between GenAI, people, and societal structures. Yet a model is rarely explicitly defined in societal impact evaluations, or in the taxonomies of societal impacts that support them. In this provocation, we argue that societal impacts should be conceptualised as application- and context-specific, incommensurable, and shaped by questions of social power. Doing so leads us to conclude that societal impact evaluations using existing taxonomies are inherently limited, in terms of their potential to reveal how GenAI systems may interact with people when introduced into specific social contexts. We therefore propose a governance-first approach to managing societal harms attended by GenAI technologies. |
| title | Troubling Taxonomies in GenAI Evaluation |
| topic | Human-Computer Interaction |
| url | https://arxiv.org/abs/2410.22985 |