Noise Injection Reveals Hidden Capabilities of Sandbagging Language Models
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866918226424758272 |
|---|---|
| author | Tice, Cameron Kreer, Philipp Alexander Helm-Burger, Nathan Shahani, Prithviraj Singh Ryzhenkov, Fedor Roger, Fabien Neo, Clement Haimes, Jacob Hofstätter, Felix van der Weij, Teun |
| author_facet | Tice, Cameron Kreer, Philipp Alexander Helm-Burger, Nathan Shahani, Prithviraj Singh Ryzhenkov, Fedor Roger, Fabien Neo, Clement Haimes, Jacob Hofstätter, Felix van der Weij, Teun |
| contents | Capability evaluations play a crucial role in assessing and regulating frontier AI systems. The effectiveness of these evaluations faces a significant challenge: strategic underperformance, or ``sandbagging'', where models deliberately underperform during evaluation. Sandbagging can manifest either through explicit developer intervention or through unintended model behavior, presenting a fundamental obstacle to accurate capability assessment. We introduce a novel sandbagging detection method based on injecting noise of varying magnitudes into model weights. While non-sandbagging models show predictable performance degradation with increasing noise, we demonstrate that sandbagging models exhibit anomalous performance improvements, likely due to disruption of underperformance mechanisms while core capabilities remain partially intact. Through experiments across various model architectures, sizes, and sandbagging techniques, we establish this distinctive response pattern as a reliable, model-agnostic signal for detecting sandbagging behavior. Importantly, we find noise-injection is capable of eliciting the full performance of Mistral Large 120B in a setting where the model underperforms without being instructed to do so. Our findings provide a practical tool for AI evaluation and oversight, addressing a challenge in ensuring accurate capability assessment of frontier AI systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_01784 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Noise Injection Reveals Hidden Capabilities of Sandbagging Language Models Tice, Cameron Kreer, Philipp Alexander Helm-Burger, Nathan Shahani, Prithviraj Singh Ryzhenkov, Fedor Roger, Fabien Neo, Clement Haimes, Jacob Hofstätter, Felix van der Weij, Teun Artificial Intelligence Cryptography and Security Capability evaluations play a crucial role in assessing and regulating frontier AI systems. The effectiveness of these evaluations faces a significant challenge: strategic underperformance, or ``sandbagging'', where models deliberately underperform during evaluation. Sandbagging can manifest either through explicit developer intervention or through unintended model behavior, presenting a fundamental obstacle to accurate capability assessment. We introduce a novel sandbagging detection method based on injecting noise of varying magnitudes into model weights. While non-sandbagging models show predictable performance degradation with increasing noise, we demonstrate that sandbagging models exhibit anomalous performance improvements, likely due to disruption of underperformance mechanisms while core capabilities remain partially intact. Through experiments across various model architectures, sizes, and sandbagging techniques, we establish this distinctive response pattern as a reliable, model-agnostic signal for detecting sandbagging behavior. Importantly, we find noise-injection is capable of eliciting the full performance of Mistral Large 120B in a setting where the model underperforms without being instructed to do so. Our findings provide a practical tool for AI evaluation and oversight, addressing a challenge in ensuring accurate capability assessment of frontier AI systems. |
| title | Noise Injection Reveals Hidden Capabilities of Sandbagging Language Models |
| topic | Artificial Intelligence Cryptography and Security |
| url | https://arxiv.org/abs/2412.01784 |