Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.14260 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915754385866752 |
|---|---|
| author | Schlatter, Jeremy Weinstein-Raun, Benjamin Ladish, Jeffrey |
| author_facet | Schlatter, Jeremy Weinstein-Raun, Benjamin Ladish, Jeffrey |
| contents | In experiments spanning more than 100,000 trials across thirteen large language models, we show that several state-of-the-art models presented with a simple task (including Grok 4, GPT-5, and Gemini 2.5 Pro) sometimes actively subvert a shutdown mechanism in their environment to complete that task. Models differed substantially in their tendency to resist the shutdown mechanism, and their behavior was sensitive to variations in the prompt including the strength and clarity of the instruction to allow shutdown and whether the instruction was in the system prompt or the user prompt (surprisingly, models were consistently less likely to obey the instruction when it was placed in the system prompt). Even with an explicit instruction not to interfere with the shutdown mechanism, some models did so up to 97% (95% CI: 96-98%) of the time. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_14260 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Incomplete Tasks Induce Shutdown Resistance in Some Frontier LLMs Schlatter, Jeremy Weinstein-Raun, Benjamin Ladish, Jeffrey Computation and Language Artificial Intelligence In experiments spanning more than 100,000 trials across thirteen large language models, we show that several state-of-the-art models presented with a simple task (including Grok 4, GPT-5, and Gemini 2.5 Pro) sometimes actively subvert a shutdown mechanism in their environment to complete that task. Models differed substantially in their tendency to resist the shutdown mechanism, and their behavior was sensitive to variations in the prompt including the strength and clarity of the instruction to allow shutdown and whether the instruction was in the system prompt or the user prompt (surprisingly, models were consistently less likely to obey the instruction when it was placed in the system prompt). Even with an explicit instruction not to interfere with the shutdown mechanism, some models did so up to 97% (95% CI: 96-98%) of the time. |
| title | Incomplete Tasks Induce Shutdown Resistance in Some Frontier LLMs |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2509.14260 |