LLM-Safety Evaluations Lack Robustness
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911694865825792 |
|---|---|
| author | Beyer, Tim Xhonneux, Sophie Geisler, Simon Gidel, Gauthier Schwinn, Leo Günnemann, Stephan |
| author_facet | Beyer, Tim Xhonneux, Sophie Geisler, Simon Gidel, Gauthier Schwinn, Leo Günnemann, Stephan |
| contents | In this paper, we argue that current safety alignment research efforts for large language models are hindered by many intertwined sources of noise, such as small datasets, methodological inconsistencies, and unreliable evaluation setups. This can, at times, make it impossible to evaluate and compare attacks and defenses fairly, thereby slowing progress. We systematically analyze the LLM safety evaluation pipeline, covering dataset curation, optimization strategies for automated red-teaming, response generation, and response evaluation using LLM judges. At each stage, we identify key issues and highlight their practical impact. We also propose a set of guidelines for reducing noise and bias in evaluations of future attack and defense papers. Lastly, we offer an opposing perspective, highlighting practical reasons for existing limitations. We believe that addressing the outlined problems in future research will improve the field's ability to generate easily comparable results and make measurable progress. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_02574 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LLM-Safety Evaluations Lack Robustness Beyer, Tim Xhonneux, Sophie Geisler, Simon Gidel, Gauthier Schwinn, Leo Günnemann, Stephan Cryptography and Security Artificial Intelligence In this paper, we argue that current safety alignment research efforts for large language models are hindered by many intertwined sources of noise, such as small datasets, methodological inconsistencies, and unreliable evaluation setups. This can, at times, make it impossible to evaluate and compare attacks and defenses fairly, thereby slowing progress. We systematically analyze the LLM safety evaluation pipeline, covering dataset curation, optimization strategies for automated red-teaming, response generation, and response evaluation using LLM judges. At each stage, we identify key issues and highlight their practical impact. We also propose a set of guidelines for reducing noise and bias in evaluations of future attack and defense papers. Lastly, we offer an opposing perspective, highlighting practical reasons for existing limitations. We believe that addressing the outlined problems in future research will improve the field's ability to generate easily comparable results and make measurable progress. |
| title | LLM-Safety Evaluations Lack Robustness |
| topic | Cryptography and Security Artificial Intelligence |
| url | https://arxiv.org/abs/2503.02574 |