Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910763565711360 |
|---|---|
| author | Ren, Richard Basart, Steven Khoja, Adam Gatti, Alice Phan, Long Yin, Xuwang Mazeika, Mantas Pan, Alexander Mukobi, Gabriel Kim, Ryan H. Fitz, Stephen Hendrycks, Dan |
| author_facet | Ren, Richard Basart, Steven Khoja, Adam Gatti, Alice Phan, Long Yin, Xuwang Mazeika, Mantas Pan, Alexander Mukobi, Gabriel Kim, Ryan H. Fitz, Stephen Hendrycks, Dan |
| contents | As artificial intelligence systems grow more powerful, there has been increasing interest in "AI safety" research to address emerging and future risks. However, the field of AI safety remains poorly defined and inconsistently measured, leading to confusion about how researchers can contribute. This lack of clarity is compounded by the unclear relationship between AI safety benchmarks and upstream general capabilities (e.g., general knowledge and reasoning). To address these issues, we conduct a comprehensive meta-analysis of AI safety benchmarks, empirically analyzing their correlation with general capabilities across dozens of models and providing a survey of existing directions in AI safety. Our findings reveal that many safety benchmarks highly correlate with both upstream model capabilities and training compute, potentially enabling "safetywashing"--where capability improvements are misrepresented as safety advancements. Based on these findings, we propose an empirical foundation for developing more meaningful safety metrics and define AI safety in a machine learning research context as a set of clearly delineated research goals that are empirically separable from generic capabilities advancements. In doing so, we aim to provide a more rigorous framework for AI safety research, advancing the science of safety evaluations and clarifying the path towards measurable progress. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_21792 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress? Ren, Richard Basart, Steven Khoja, Adam Gatti, Alice Phan, Long Yin, Xuwang Mazeika, Mantas Pan, Alexander Mukobi, Gabriel Kim, Ryan H. Fitz, Stephen Hendrycks, Dan Machine Learning Artificial Intelligence Computation and Language Computers and Society As artificial intelligence systems grow more powerful, there has been increasing interest in "AI safety" research to address emerging and future risks. However, the field of AI safety remains poorly defined and inconsistently measured, leading to confusion about how researchers can contribute. This lack of clarity is compounded by the unclear relationship between AI safety benchmarks and upstream general capabilities (e.g., general knowledge and reasoning). To address these issues, we conduct a comprehensive meta-analysis of AI safety benchmarks, empirically analyzing their correlation with general capabilities across dozens of models and providing a survey of existing directions in AI safety. Our findings reveal that many safety benchmarks highly correlate with both upstream model capabilities and training compute, potentially enabling "safetywashing"--where capability improvements are misrepresented as safety advancements. Based on these findings, we propose an empirical foundation for developing more meaningful safety metrics and define AI safety in a machine learning research context as a set of clearly delineated research goals that are empirically separable from generic capabilities advancements. In doing so, we aim to provide a more rigorous framework for AI safety research, advancing the science of safety evaluations and clarifying the path towards measurable progress. |
| title | Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress? |
| topic | Machine Learning Artificial Intelligence Computation and Language Computers and Society |
| url | https://arxiv.org/abs/2407.21792 |