Comparison requires valid measurement: Rethinking attack success rate comparisons in AI red teaming
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866918305043841024 |
|---|---|
| author | Chouldechova, Alexandra Cooper, A. Feder Barocas, Solon Palia, Abhinav Vann, Dan Wallach, Hanna |
| author_facet | Chouldechova, Alexandra Cooper, A. Feder Barocas, Solon Palia, Abhinav Vann, Dan Wallach, Hanna |
| contents | We argue that conclusions drawn about relative system safety or attack method efficacy via AI red teaming are often not supported by evidence provided by attack success rate (ASR) comparisons. We show, through conceptual, theoretical, and empirical contributions, that many conclusions are founded on apples-to-oranges comparisons or low-validity measurements. Our arguments are grounded in asking a simple question: When can attack success rates be meaningfully compared? To answer this question, we draw on ideas from social science measurement theory and inferential statistics, which, taken together, provide a conceptual grounding for understanding when numerical values obtained through the quantification of system attributes can be meaningfully compared. Through this lens, we articulate conditions under which ASRs can and cannot be meaningfully compared. Using jailbreaking as a running example, we provide examples and extensive discussion of apples-to-oranges ASR comparisons and measurement validity challenges. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_18076 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Comparison requires valid measurement: Rethinking attack success rate comparisons in AI red teaming Chouldechova, Alexandra Cooper, A. Feder Barocas, Solon Palia, Abhinav Vann, Dan Wallach, Hanna Machine Learning We argue that conclusions drawn about relative system safety or attack method efficacy via AI red teaming are often not supported by evidence provided by attack success rate (ASR) comparisons. We show, through conceptual, theoretical, and empirical contributions, that many conclusions are founded on apples-to-oranges comparisons or low-validity measurements. Our arguments are grounded in asking a simple question: When can attack success rates be meaningfully compared? To answer this question, we draw on ideas from social science measurement theory and inferential statistics, which, taken together, provide a conceptual grounding for understanding when numerical values obtained through the quantification of system attributes can be meaningfully compared. Through this lens, we articulate conditions under which ASRs can and cannot be meaningfully compared. Using jailbreaking as a running example, we provide examples and extensive discussion of apples-to-oranges ASR comparisons and measurement validity challenges. |
| title | Comparison requires valid measurement: Rethinking attack success rate comparisons in AI red teaming |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2601.18076 |