Copertina

Comparison requires valid measurement: Rethinking attack success rate comparisons in AI red teaming

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Chouldechova, Alexandra, Cooper, A. Feder, Barocas, Solon, Palia, Abhinav, Vann, Dan, Wallach, Hanna
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

A Framework for Evaluating LLMs Under Task Indeterminacy
di: Guerdan, Luke, et al.
Pubblicazione: (2024)

Validating LLM-as-a-Judge Systems under Rating Indeterminacy
di: Guerdan, Luke, et al.
Pubblicazione: (2025)

Dimensions of Generative AI Evaluation Design
di: Dow, P. Alex, et al.
Pubblicazione: (2024)

Effects of Generative AI Errors on User Reliance Across Task Difficulty
di: Anthis, Jacy Reese, et al.
Pubblicazione: (2026)

A Shared Standard for Valid Measurement of Generative AI Systems' Capabilities, Risks, and Impacts
di: Chouldechova, Alexandra, et al.
Pubblicazione: (2024)

AI-Assisted Systematization for Evaluating GenAI Systems
di: Agarwal, Dhruv, et al.
Pubblicazione: (2026)

Evaluating Generative AI Systems is a Social Science Measurement Challenge
di: Wallach, Hanna, et al.
Pubblicazione: (2024)

Position: Evaluating Generative AI Systems Is a Social Science Measurement Challenge
di: Wallach, Hanna, et al.
Pubblicazione: (2025)

What Constitutes a Less Discriminatory Algorithm?
di: Laufer, Benjamin, et al.
Pubblicazione: (2024)

Statistical Guarantees in the Search for Less Discriminatory Algorithms
di: Hays, Chris, et al.
Pubblicazione: (2025)

Arbitrariness and Social Prediction: The Confounding Role of Variance in Fair Classification
di: Cooper, A. Feder, et al.
Pubblicazione: (2023)

Between Randomness and Arbitrariness: Some Lessons for Reliable Machine Learning at Scale
di: Cooper, A. Feder
Pubblicazione: (2024)

Learning diverse attacks on large language models for robust red-teaming and safety tuning
di: Lee, Seanie, et al.
Pubblicazione: (2024)

Distinguishing Task-Specific and General-Purpose AI in Regulation
di: Wang, Jennifer, et al.
Pubblicazione: (2025)

Rigor in AI: Doing Rigorous AI Work Requires a Broader, Responsible AI-Informed Conception of Rigor
di: Olteanu, Alexandra, et al.
Pubblicazione: (2025)

Taxonomizing Representational Harms using Speech Act Theory
di: Corvi, Emily, et al.
Pubblicazione: (2025)

Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based Systems
di: Harvey, Emma, et al.
Pubblicazione: (2024)

Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems
di: Harvey, Emma, et al.
Pubblicazione: (2025)

The Impact of Differential Feature Under-reporting on Algorithmic Fairness
di: Akpinar, Nil-Jana, et al.
Pubblicazione: (2024)

A structured regression approach for evaluating model performance across intersectional subgroups
di: Herlihy, Christine, et al.
Pubblicazione: (2024)

SureMap: Simultaneous Mean Estimation for Single-Task and Multi-Task Disaggregated Evaluation
di: Khodak, Mikhail, et al.
Pubblicazione: (2024)

AI Automatons: AI Systems Intended to Imitate Humans
di: Olteanu, Alexandra, et al.
Pubblicazione: (2025)

Machine Unlearning Doesn't Do What You Think: Lessons for Generative AI Policy and Research
di: Cooper, A. Feder, et al.
Pubblicazione: (2024)

Leveraging Expert Consistency to Improve Algorithmic Decision Support
di: De-Arteaga, Maria, et al.
Pubblicazione: (2021)

Designing Algorithmic Delegates: The Role of Indistinguishability in Human-AI Handoff
di: Greenwood, Sophie, et al.
Pubblicazione: (2025)

Non-Determinism and the Lawlessness of Machine Learning Code
di: Cooper, A. Feder, et al.
Pubblicazione: (2022)

Rethinking external validation for the target population: Capturing patient-level similarity with a generative model
di: Azizmalayeri, Mohammad, et al.
Pubblicazione: (2026)

Supporting Industry Computing Researchers in Assessing, Articulating, and Addressing the Potential Negative Societal Impact of Their Work
di: Deng, Wesley Hanwen, et al.
Pubblicazione: (2024)

The Files are in the Computer: On Copyright, Memorization, and Generative AI
di: Cooper, A. Feder, et al.
Pubblicazione: (2024)

Do Responsible AI Artifacts Advance Stakeholder Goals? Four Key Barriers Perceived by Legal and Civil Stakeholders
di: Kawakami, Anna, et al.
Pubblicazione: (2024)

Extracting books from production language models
di: Ahmed, Ahmed, et al.
Pubblicazione: (2026)

Measuring Machine Learning Harms from Stereotypes Requires Understanding Who Is Harmed by Which Errors in What Ways
di: Wang, Angelina, et al.
Pubblicazione: (2024)

A team of teams creates success from successes
di: Stan Skipworth
Pubblicazione: (2024)

A team of teams creates success from successes
di: Stan Skipworth
Pubblicazione: (2024)

Exploring the limits of strong membership inference attacks on large language models
di: Hayes, Jamie, et al.
Pubblicazione: (2025)

Error Exponent in Agnostic PAC Learning
di: Hendel, Adi, et al.
Pubblicazione: (2024)

Toward an Evaluation Science for Generative AI Systems
di: Weidinger, Laura, et al.
Pubblicazione: (2025)

Talkin' 'Bout AI Generation: Copyright and the Generative-AI Supply Chain
di: Lee, Katherine, et al.
Pubblicazione: (2023)

Universal Batch Learning Under The Misspecification Setting
di: Vituri, Shlomi, et al.
Pubblicazione: (2024)

Minimax rates of convergence for nonparametric regression under adversarial attacks
di: Peng, Jingfu, et al.
Pubblicazione: (2024)