Is ETHICS about ethics? Evaluating the ETHICS benchmark
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916495776284672 |
|---|---|
| author | Hancox-Li, Leif Blili-Hamelin, Borhane |
| author_facet | Hancox-Li, Leif Blili-Hamelin, Borhane |
| contents | ETHICS is probably the most-cited dataset for testing the ethical capabilities of language models. Drawing on moral theory, psychology, and prompt evaluation, we interrogate the validity of the ETHICS benchmark. Adding to prior work, our findings suggest that having a clear understanding of ethics and how it relates to empirical phenomena is key to the validity of ethics evaluations for AI. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_13009 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Is ETHICS about ethics? Evaluating the ETHICS benchmark Hancox-Li, Leif Blili-Hamelin, Borhane Computers and Society ETHICS is probably the most-cited dataset for testing the ethical capabilities of language models. Drawing on moral theory, psychology, and prompt evaluation, we interrogate the validity of the ETHICS benchmark. Adding to prior work, our findings suggest that having a clear understanding of ethics and how it relates to empirical phenomena is key to the validity of ethics evaluations for AI. |
| title | Is ETHICS about ethics? Evaluating the ETHICS benchmark |
| topic | Computers and Society |
| url | https://arxiv.org/abs/2410.13009 |