STREAM (ChemBio): A Standard for Transparently Reporting Evaluations in AI Model Reports
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918134772924416 |
|---|---|
| author | McCaslin, Tegan Alaga, Jide Nedungadi, Samira Donoughe, Seth Reed, Tom Bommasani, Rishi Painter, Chris Righetti, Luca |
| author_facet | McCaslin, Tegan Alaga, Jide Nedungadi, Samira Donoughe, Seth Reed, Tom Bommasani, Rishi Painter, Chris Righetti, Luca |
| contents | Evaluations of dangerous AI capabilities are important for managing catastrophic risks. Public transparency into these evaluations - including what they test, how they are conducted, and how their results inform decisions - is crucial for building trust in AI development. We propose STREAM (A Standard for Transparently Reporting Evaluations in AI Model Reports), a standard to improve how model reports disclose evaluation results, initially focusing on chemical and biological (ChemBio) benchmarks. Developed in consultation with 23 experts across government, civil society, academia, and frontier AI companies, this standard is designed to (1) be a practical resource to help AI developers present evaluation results more clearly, and (2) help third parties identify whether model reports provide sufficient detail to assess the rigor of the ChemBio evaluations. We concretely demonstrate our proposed best practices with "gold standard" examples, and also provide a three-page reporting template to enable AI developers to implement our recommendations more easily. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_09853 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | STREAM (ChemBio): A Standard for Transparently Reporting Evaluations in AI Model Reports McCaslin, Tegan Alaga, Jide Nedungadi, Samira Donoughe, Seth Reed, Tom Bommasani, Rishi Painter, Chris Righetti, Luca Computers and Society Artificial Intelligence Evaluations of dangerous AI capabilities are important for managing catastrophic risks. Public transparency into these evaluations - including what they test, how they are conducted, and how their results inform decisions - is crucial for building trust in AI development. We propose STREAM (A Standard for Transparently Reporting Evaluations in AI Model Reports), a standard to improve how model reports disclose evaluation results, initially focusing on chemical and biological (ChemBio) benchmarks. Developed in consultation with 23 experts across government, civil society, academia, and frontier AI companies, this standard is designed to (1) be a practical resource to help AI developers present evaluation results more clearly, and (2) help third parties identify whether model reports provide sufficient detail to assess the rigor of the ChemBio evaluations. We concretely demonstrate our proposed best practices with "gold standard" examples, and also provide a three-page reporting template to enable AI developers to implement our recommendations more easily. |
| title | STREAM (ChemBio): A Standard for Transparently Reporting Evaluations in AI Model Reports |
| topic | Computers and Society Artificial Intelligence |
| url | https://arxiv.org/abs/2508.09853 |