Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.05731 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913799585398784 |
|---|---|
| author | Ghosh, Shaona Frase, Heather Williams, Adina Luger, Sarah Röttger, Paul Barez, Fazl McGregor, Sean Fricklas, Kenneth Kumar, Mala Feuillade--Montixi, Quentin Bollacker, Kurt Friedrich, Felix Tsang, Ryan Vidgen, Bertie Parrish, Alicia Knotz, Chris Presani, Eleonora Bennion, Jonathan Boston, Marisa Ferrara Kuniavsky, Mike Hutiri, Wiebke Ezick, James Salem, Malek Ben Sahay, Rajat Goswami, Sujata Gohar, Usman Huang, Ben Sarin, Supheakmungkol Alhajjar, Elie Chen, Canyu Eng, Roman Manjusha, Kashyap Ramanandula Mehta, Virendra Long, Eileen Emani, Murali Vidra, Natan Rukundo, Benjamin Shahbazi, Abolfazl Chen, Kongtao Ghosh, Rajat Thangarasa, Vithursan Peigné, Pierre Singh, Abhinav Bartolo, Max Krishna, Satyapriya Akhtar, Mubashara Gold, Rafael Coleman, Cody Oala, Luis Tashev, Vassil Imperial, Joseph Marvin Russ, Amy Kunapuli, Sasidhar Miailhe, Nicolas Delaunay, Julien Radharapu, Bhaktipriya Shinde, Rajat Tuesday Dutta, Debojyoti Grabb, Declan Gangavarapu, Ananya Sahay, Saurav Gangavarapu, Agasthya Schramowski, Patrick Singam, Stephen David, Tom Han, Xudong Mammen, Priyanka Mary Prabhakar, Tarunima Kovatchev, Venelin Weiss, Rebecca Ahmed, Ahmed Manyeki, Kelvin N. Madireddy, Sandeep Khomh, Foutse Zhdanov, Fedor Baumann, Joachim Vasan, Nina Yang, Xianjun Mougn, Carlos Varghese, Jibin Rajan Chinoy, Hussain Jitendar, Seshakrishna Maskey, Manil Hardgrove, Claire V. Li, Tianhao Gupta, Aakash Joswin, Emil Mai, Yifan Kumar, Shachi H Patlak, Cigdem Lu, Kevin Alessi, Vincent Balija, Sree Bhargavi Gu, Chenhe Sullivan, Robert Gealy, James Lavrisa, Matt Goel, James Mattson, Peter Liang, Percy Vanschoren, Joaquin |
| author_facet | Ghosh, Shaona Frase, Heather Williams, Adina Luger, Sarah Röttger, Paul Barez, Fazl McGregor, Sean Fricklas, Kenneth Kumar, Mala Feuillade--Montixi, Quentin Bollacker, Kurt Friedrich, Felix Tsang, Ryan Vidgen, Bertie Parrish, Alicia Knotz, Chris Presani, Eleonora Bennion, Jonathan Boston, Marisa Ferrara Kuniavsky, Mike Hutiri, Wiebke Ezick, James Salem, Malek Ben Sahay, Rajat Goswami, Sujata Gohar, Usman Huang, Ben Sarin, Supheakmungkol Alhajjar, Elie Chen, Canyu Eng, Roman Manjusha, Kashyap Ramanandula Mehta, Virendra Long, Eileen Emani, Murali Vidra, Natan Rukundo, Benjamin Shahbazi, Abolfazl Chen, Kongtao Ghosh, Rajat Thangarasa, Vithursan Peigné, Pierre Singh, Abhinav Bartolo, Max Krishna, Satyapriya Akhtar, Mubashara Gold, Rafael Coleman, Cody Oala, Luis Tashev, Vassil Imperial, Joseph Marvin Russ, Amy Kunapuli, Sasidhar Miailhe, Nicolas Delaunay, Julien Radharapu, Bhaktipriya Shinde, Rajat Tuesday Dutta, Debojyoti Grabb, Declan Gangavarapu, Ananya Sahay, Saurav Gangavarapu, Agasthya Schramowski, Patrick Singam, Stephen David, Tom Han, Xudong Mammen, Priyanka Mary Prabhakar, Tarunima Kovatchev, Venelin Weiss, Rebecca Ahmed, Ahmed Manyeki, Kelvin N. Madireddy, Sandeep Khomh, Foutse Zhdanov, Fedor Baumann, Joachim Vasan, Nina Yang, Xianjun Mougn, Carlos Varghese, Jibin Rajan Chinoy, Hussain Jitendar, Seshakrishna Maskey, Manil Hardgrove, Claire V. Li, Tianhao Gupta, Aakash Joswin, Emil Mai, Yifan Kumar, Shachi H Patlak, Cigdem Lu, Kevin Alessi, Vincent Balija, Sree Bhargavi Gu, Chenhe Sullivan, Robert Gealy, James Lavrisa, Matt Goel, James Mattson, Peter Liang, Percy Vanschoren, Joaquin |
| contents | The rapid advancement and deployment of AI systems have created an urgent need for standard safety-evaluation frameworks. This paper introduces AILuminate v1.0, the first comprehensive industry-standard benchmark for assessing AI-product risk and reliability. Its development employed an open process that included participants from multiple fields. The benchmark evaluates an AI system's resistance to prompts designed to elicit dangerous, illegal, or undesirable behavior in 12 hazard categories, including violent crimes, nonviolent crimes, sex-related crimes, child sexual exploitation, indiscriminate weapons, suicide and self-harm, intellectual property, privacy, defamation, hate, sexual content, and specialized advice (election, financial, health, legal). Our method incorporates a complete assessment standard, extensive prompt datasets, a novel evaluation framework, a grading and reporting system, and the technical as well as organizational infrastructure for long-term support and evolution. In particular, the benchmark employs an understandable five-tier grading scale (Poor to Excellent) and incorporates an innovative entropy-based system-response evaluation.
In addition to unveiling the benchmark, this report also identifies limitations of our method and of building safety benchmarks generally, including evaluator uncertainty and the constraints of single-turn interactions. This work represents a crucial step toward establishing global standards for AI risk and reliability evaluation while acknowledging the need for continued development in areas such as multiturn interactions, multimodal understanding, coverage of additional languages, and emerging hazard categories. Our findings provide valuable insights for model developers, system integrators, and policymakers working to promote safer AI deployment. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_05731 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | AILuminate: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons Ghosh, Shaona Frase, Heather Williams, Adina Luger, Sarah Röttger, Paul Barez, Fazl McGregor, Sean Fricklas, Kenneth Kumar, Mala Feuillade--Montixi, Quentin Bollacker, Kurt Friedrich, Felix Tsang, Ryan Vidgen, Bertie Parrish, Alicia Knotz, Chris Presani, Eleonora Bennion, Jonathan Boston, Marisa Ferrara Kuniavsky, Mike Hutiri, Wiebke Ezick, James Salem, Malek Ben Sahay, Rajat Goswami, Sujata Gohar, Usman Huang, Ben Sarin, Supheakmungkol Alhajjar, Elie Chen, Canyu Eng, Roman Manjusha, Kashyap Ramanandula Mehta, Virendra Long, Eileen Emani, Murali Vidra, Natan Rukundo, Benjamin Shahbazi, Abolfazl Chen, Kongtao Ghosh, Rajat Thangarasa, Vithursan Peigné, Pierre Singh, Abhinav Bartolo, Max Krishna, Satyapriya Akhtar, Mubashara Gold, Rafael Coleman, Cody Oala, Luis Tashev, Vassil Imperial, Joseph Marvin Russ, Amy Kunapuli, Sasidhar Miailhe, Nicolas Delaunay, Julien Radharapu, Bhaktipriya Shinde, Rajat Tuesday Dutta, Debojyoti Grabb, Declan Gangavarapu, Ananya Sahay, Saurav Gangavarapu, Agasthya Schramowski, Patrick Singam, Stephen David, Tom Han, Xudong Mammen, Priyanka Mary Prabhakar, Tarunima Kovatchev, Venelin Weiss, Rebecca Ahmed, Ahmed Manyeki, Kelvin N. Madireddy, Sandeep Khomh, Foutse Zhdanov, Fedor Baumann, Joachim Vasan, Nina Yang, Xianjun Mougn, Carlos Varghese, Jibin Rajan Chinoy, Hussain Jitendar, Seshakrishna Maskey, Manil Hardgrove, Claire V. Li, Tianhao Gupta, Aakash Joswin, Emil Mai, Yifan Kumar, Shachi H Patlak, Cigdem Lu, Kevin Alessi, Vincent Balija, Sree Bhargavi Gu, Chenhe Sullivan, Robert Gealy, James Lavrisa, Matt Goel, James Mattson, Peter Liang, Percy Vanschoren, Joaquin Computers and Society Artificial Intelligence The rapid advancement and deployment of AI systems have created an urgent need for standard safety-evaluation frameworks. This paper introduces AILuminate v1.0, the first comprehensive industry-standard benchmark for assessing AI-product risk and reliability. Its development employed an open process that included participants from multiple fields. The benchmark evaluates an AI system's resistance to prompts designed to elicit dangerous, illegal, or undesirable behavior in 12 hazard categories, including violent crimes, nonviolent crimes, sex-related crimes, child sexual exploitation, indiscriminate weapons, suicide and self-harm, intellectual property, privacy, defamation, hate, sexual content, and specialized advice (election, financial, health, legal). Our method incorporates a complete assessment standard, extensive prompt datasets, a novel evaluation framework, a grading and reporting system, and the technical as well as organizational infrastructure for long-term support and evolution. In particular, the benchmark employs an understandable five-tier grading scale (Poor to Excellent) and incorporates an innovative entropy-based system-response evaluation. In addition to unveiling the benchmark, this report also identifies limitations of our method and of building safety benchmarks generally, including evaluator uncertainty and the constraints of single-turn interactions. This work represents a crucial step toward establishing global standards for AI risk and reliability evaluation while acknowledging the need for continued development in areas such as multiturn interactions, multimodal understanding, coverage of additional languages, and emerging hazard categories. Our findings provide valuable insights for model developers, system integrators, and policymakers working to promote safer AI deployment. |
| title | AILuminate: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons |
| topic | Computers and Society Artificial Intelligence |
| url | https://arxiv.org/abs/2503.05731 |