Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2506.06414 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910150655213568 |
|---|---|
| author | Brown, Davis Sabbaghi, Mahdi Sun, Luze Robey, Alexander Pappas, George J. Wong, Eric Hassani, Hamed |
| author_facet | Brown, Davis Sabbaghi, Mahdi Sun, Luze Robey, Alexander Pappas, George J. Wong, Eric Hassani, Hamed |
| contents | Existing language model safety evaluations focus on overt attacks and low-stakes tasks. In reality, an attacker can easily subvert existing safeguards by requesting help on small, benign-seeming tasks across many independent queries. Because the individual queries do not appear harmful, the attack is hard to detect. However, when combined, these fragments uplift misuse by helping the attacker complete hard and dangerous tasks. Toward identifying defenses against such strategies, we develop Benchmarks for Stateful Defenses (BSD), a data generation pipeline that automates evaluations of covert attacks and corresponding defenses. Using this pipeline, we curate two new datasets that are consistently refused by frontier models and are too difficult for weaker open-weight models. This enables us to evaluate decomposition attacks, which are found to be effective misuse enablers, and to highlight stateful defenses as a promising countermeasure. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_06414 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Benchmarking Misuse Mitigation Against Covert Adversaries Brown, Davis Sabbaghi, Mahdi Sun, Luze Robey, Alexander Pappas, George J. Wong, Eric Hassani, Hamed Cryptography and Security Artificial Intelligence Existing language model safety evaluations focus on overt attacks and low-stakes tasks. In reality, an attacker can easily subvert existing safeguards by requesting help on small, benign-seeming tasks across many independent queries. Because the individual queries do not appear harmful, the attack is hard to detect. However, when combined, these fragments uplift misuse by helping the attacker complete hard and dangerous tasks. Toward identifying defenses against such strategies, we develop Benchmarks for Stateful Defenses (BSD), a data generation pipeline that automates evaluations of covert attacks and corresponding defenses. Using this pipeline, we curate two new datasets that are consistently refused by frontier models and are too difficult for weaker open-weight models. This enables us to evaluate decomposition attacks, which are found to be effective misuse enablers, and to highlight stateful defenses as a promising countermeasure. |
| title | Benchmarking Misuse Mitigation Against Covert Adversaries |
| topic | Cryptography and Security Artificial Intelligence |
| url | https://arxiv.org/abs/2506.06414 |