Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.16861 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916661690368000 |
|---|---|
| author | Longpre, Shayne Klyman, Kevin Appel, Ruth E. Kapoor, Sayash Bommasani, Rishi Sahar, Michelle McGregor, Sean Ghosh, Avijit Blili-Hamelin, Borhane Butters, Nathan Nelson, Alondra Elazari, Amit Sellars, Andrew Ellis, Casey John Sherrets, Dane Song, Dawn Geiger, Harley Cohen, Ilona McIlvenny, Lauren Srikumar, Madhulika Jaycox, Mark M. Anderljung, Markus Johnson, Nadine Farid Carlini, Nicholas Miailhe, Nicolas Marda, Nik Henderson, Peter Portnoff, Rebecca S. Weiss, Rebecca Westerhoff, Victoria Jernite, Yacine Chowdhury, Rumman Liang, Percy Narayanan, Arvind |
| author_facet | Longpre, Shayne Klyman, Kevin Appel, Ruth E. Kapoor, Sayash Bommasani, Rishi Sahar, Michelle McGregor, Sean Ghosh, Avijit Blili-Hamelin, Borhane Butters, Nathan Nelson, Alondra Elazari, Amit Sellars, Andrew Ellis, Casey John Sherrets, Dane Song, Dawn Geiger, Harley Cohen, Ilona McIlvenny, Lauren Srikumar, Madhulika Jaycox, Mark M. Anderljung, Markus Johnson, Nadine Farid Carlini, Nicholas Miailhe, Nicolas Marda, Nik Henderson, Peter Portnoff, Rebecca S. Weiss, Rebecca Westerhoff, Victoria Jernite, Yacine Chowdhury, Rumman Liang, Percy Narayanan, Arvind |
| contents | The widespread deployment of general-purpose AI (GPAI) systems introduces significant new risks. Yet the infrastructure, practices, and norms for reporting flaws in GPAI systems remain seriously underdeveloped, lagging far behind more established fields like software security. Based on a collaboration between experts from the fields of software security, machine learning, law, social science, and policy, we identify key gaps in the evaluation and reporting of flaws in GPAI systems. We call for three interventions to advance system safety. First, we propose using standardized AI flaw reports and rules of engagement for researchers in order to ease the process of submitting, reproducing, and triaging flaws in GPAI systems. Second, we propose GPAI system providers adopt broadly-scoped flaw disclosure programs, borrowing from bug bounties, with legal safe harbors to protect researchers. Third, we advocate for the development of improved infrastructure to coordinate distribution of flaw reports across the many stakeholders who may be impacted. These interventions are increasingly urgent, as evidenced by the prevalence of jailbreaks and other flaws that can transfer across different providers' GPAI systems. By promoting robust reporting and coordination in the AI ecosystem, these proposals could significantly improve the safety, security, and accountability of GPAI systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_16861 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | In-House Evaluation Is Not Enough: Towards Robust Third-Party Flaw Disclosure for General-Purpose AI Longpre, Shayne Klyman, Kevin Appel, Ruth E. Kapoor, Sayash Bommasani, Rishi Sahar, Michelle McGregor, Sean Ghosh, Avijit Blili-Hamelin, Borhane Butters, Nathan Nelson, Alondra Elazari, Amit Sellars, Andrew Ellis, Casey John Sherrets, Dane Song, Dawn Geiger, Harley Cohen, Ilona McIlvenny, Lauren Srikumar, Madhulika Jaycox, Mark M. Anderljung, Markus Johnson, Nadine Farid Carlini, Nicholas Miailhe, Nicolas Marda, Nik Henderson, Peter Portnoff, Rebecca S. Weiss, Rebecca Westerhoff, Victoria Jernite, Yacine Chowdhury, Rumman Liang, Percy Narayanan, Arvind Artificial Intelligence The widespread deployment of general-purpose AI (GPAI) systems introduces significant new risks. Yet the infrastructure, practices, and norms for reporting flaws in GPAI systems remain seriously underdeveloped, lagging far behind more established fields like software security. Based on a collaboration between experts from the fields of software security, machine learning, law, social science, and policy, we identify key gaps in the evaluation and reporting of flaws in GPAI systems. We call for three interventions to advance system safety. First, we propose using standardized AI flaw reports and rules of engagement for researchers in order to ease the process of submitting, reproducing, and triaging flaws in GPAI systems. Second, we propose GPAI system providers adopt broadly-scoped flaw disclosure programs, borrowing from bug bounties, with legal safe harbors to protect researchers. Third, we advocate for the development of improved infrastructure to coordinate distribution of flaw reports across the many stakeholders who may be impacted. These interventions are increasingly urgent, as evidenced by the prevalence of jailbreaks and other flaws that can transfer across different providers' GPAI systems. By promoting robust reporting and coordination in the AI ecosystem, these proposals could significantly improve the safety, security, and accountability of GPAI systems. |
| title | In-House Evaluation Is Not Enough: Towards Robust Third-Party Flaw Disclosure for General-Purpose AI |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2503.16861 |