Attack Selection Reduces Safety in Concentrated AI Control Settings against Trusted Monitoring
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911597069336576 |
|---|---|
| author | Schaeffer, Joachim Khandelwal, Arjun Tracy, Tyler |
| author_facet | Schaeffer, Joachim Khandelwal, Arjun Tracy, Tyler |
| contents | Future AI deployments will likely be monitored for malicious behaviour. The ability of these AIs to subvert monitors by adversarially selecting against them - attack selection - is particularly concerning. To study this, we let a red team create attack policies that attempt to insert attacks into code without being caught by a monitor in the concentrated BigCodeBench backdooring setting. We decompose attack selection into two problems: mapping attacks to a quality score and mapping quality scores to submission probabilities. We frame attack selection as a classification problem and show that safety is significantly more sensitive to FPR than TPR. We find that prompting the attacker model to reason about the monitor while being cautious with attack selection reduces safety from a baseline of 99% to 59% at 0.5% auditing budget, emphasizing that eliciting attack selection capabilities of models is vital to avoid overly optimistic safety scores in control evaluations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_04930 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Attack Selection Reduces Safety in Concentrated AI Control Settings against Trusted Monitoring Schaeffer, Joachim Khandelwal, Arjun Tracy, Tyler Cryptography and Security Artificial Intelligence 68T01 I.2.0 Future AI deployments will likely be monitored for malicious behaviour. The ability of these AIs to subvert monitors by adversarially selecting against them - attack selection - is particularly concerning. To study this, we let a red team create attack policies that attempt to insert attacks into code without being caught by a monitor in the concentrated BigCodeBench backdooring setting. We decompose attack selection into two problems: mapping attacks to a quality score and mapping quality scores to submission probabilities. We frame attack selection as a classification problem and show that safety is significantly more sensitive to FPR than TPR. We find that prompting the attacker model to reason about the monitor while being cautious with attack selection reduces safety from a baseline of 99% to 59% at 0.5% auditing budget, emphasizing that eliciting attack selection capabilities of models is vital to avoid overly optimistic safety scores in control evaluations. |
| title | Attack Selection Reduces Safety in Concentrated AI Control Settings against Trusted Monitoring |
| topic | Cryptography and Security Artificial Intelligence 68T01 I.2.0 |
| url | https://arxiv.org/abs/2602.04930 |