Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2505.01420 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909675030577152 |
|---|---|
| author | Phuong, Mary Zimmermann, Roland S. Wang, Ziyue Lindner, David Krakovna, Victoria Cogan, Sarah Dafoe, Allan Ho, Lewis Shah, Rohin |
| author_facet | Phuong, Mary Zimmermann, Roland S. Wang, Ziyue Lindner, David Krakovna, Victoria Cogan, Sarah Dafoe, Allan Ho, Lewis Shah, Rohin |
| contents | Recent work has demonstrated the plausibility of frontier AI models scheming -- knowingly and covertly pursuing an objective misaligned with its developer's intentions. Such behavior could be very hard to detect, and if present in future advanced systems, could pose severe loss of control risk. It is therefore important for AI developers to rule out harm from scheming prior to model deployment. In this paper, we present a suite of scheming reasoning evaluations measuring two types of reasoning capabilities that we believe are prerequisites for successful scheming: First, we propose five evaluations of ability to reason about and circumvent oversight (stealth). Second, we present eleven evaluations for measuring a model's ability to instrumentally reason about itself, its environment and its deployment (situational awareness). We demonstrate how these evaluations can be used as part of a scheming inability safety case: a model that does not succeed on these evaluations is almost certainly incapable of causing severe harm via scheming in real deployment. We run our evaluations on current frontier models and find that none of them show concerning levels of either situational awareness or stealth. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_01420 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Evaluating Frontier Models for Stealth and Situational Awareness Phuong, Mary Zimmermann, Roland S. Wang, Ziyue Lindner, David Krakovna, Victoria Cogan, Sarah Dafoe, Allan Ho, Lewis Shah, Rohin Machine Learning Recent work has demonstrated the plausibility of frontier AI models scheming -- knowingly and covertly pursuing an objective misaligned with its developer's intentions. Such behavior could be very hard to detect, and if present in future advanced systems, could pose severe loss of control risk. It is therefore important for AI developers to rule out harm from scheming prior to model deployment. In this paper, we present a suite of scheming reasoning evaluations measuring two types of reasoning capabilities that we believe are prerequisites for successful scheming: First, we propose five evaluations of ability to reason about and circumvent oversight (stealth). Second, we present eleven evaluations for measuring a model's ability to instrumentally reason about itself, its environment and its deployment (situational awareness). We demonstrate how these evaluations can be used as part of a scheming inability safety case: a model that does not succeed on these evaluations is almost certainly incapable of causing severe harm via scheming in real deployment. We run our evaluations on current frontier models and find that none of them show concerning levels of either situational awareness or stealth. |
| title | Evaluating Frontier Models for Stealth and Situational Awareness |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2505.01420 |