Beyond Benchmarks: On The False Promise of AI Regulation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909961098887168 |
|---|---|
| author | Stanovsky, Gabriel Keydar, Renana Perl, Gadi Habba, Eliya |
| author_facet | Stanovsky, Gabriel Keydar, Renana Perl, Gadi Habba, Eliya |
| contents | The performance of AI models on safety benchmarks does not indicate their real-world performance after deployment. This opaqueness of AI models impedes existing regulatory frameworks constituted on benchmark performance, leaving them incapable of mitigating ongoing real-world harm. The problem stems from a fundamental challenge in AI interpretability, which seems to be overlooked by regulators and decision makers. We propose a simple, realistic and readily usable regulatory framework which does not rely on benchmarks, and call for interdisciplinary collaboration to find new ways to address this crucial problem. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_15693 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Beyond Benchmarks: On The False Promise of AI Regulation Stanovsky, Gabriel Keydar, Renana Perl, Gadi Habba, Eliya Machine Learning Artificial Intelligence Computation and Language The performance of AI models on safety benchmarks does not indicate their real-world performance after deployment. This opaqueness of AI models impedes existing regulatory frameworks constituted on benchmark performance, leaving them incapable of mitigating ongoing real-world harm. The problem stems from a fundamental challenge in AI interpretability, which seems to be overlooked by regulators and decision makers. We propose a simple, realistic and readily usable regulatory framework which does not rely on benchmarks, and call for interdisciplinary collaboration to find new ways to address this crucial problem. |
| title | Beyond Benchmarks: On The False Promise of AI Regulation |
| topic | Machine Learning Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2501.15693 |