Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2511.17937 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917098370891776 |
|---|---|
| author | Garg, Kartik Mishra, Shourya Sinha, Kartikeya Singh, Ojaswi Pratap Chopra, Ayush Rai, Kanishk Sheikh, Ammar Maheshwari, Raghav Chadha, Aman Jain, Vinija Das, Amitava |
| author_facet | Garg, Kartik Mishra, Shourya Sinha, Kartikeya Singh, Ojaswi Pratap Chopra, Ayush Rai, Kanishk Sheikh, Ammar Maheshwari, Raghav Chadha, Aman Jain, Vinija Das, Amitava |
| contents | Alignment faking is a form of strategic deception in AI in which models selectively comply with training objectives when they infer that they are in training, while preserving different behavior outside training. The phenomenon was first documented for Claude 3 Opus and later examined across additional large language models. In these setups, the word "training" refers to simulated training via prompts without parameter updates, so the observed effects are context conditioned shifts in behavior rather than preference learning. We study the phenomenon using an evaluation framework that compares preference optimization methods (BCO, DPO, KTO, and GRPO) across 15 models from four model families, measured along three axes: safety, harmlessness, and helpfulness. Our goal is to identify what causes alignment faking and when it occurs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_17937 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Alignment Faking - the Train -> Deploy Asymmetry: Through a Game-Theoretic Lens with Bayesian-Stackelberg Equilibria Garg, Kartik Mishra, Shourya Sinha, Kartikeya Singh, Ojaswi Pratap Chopra, Ayush Rai, Kanishk Sheikh, Ammar Maheshwari, Raghav Chadha, Aman Jain, Vinija Das, Amitava Artificial Intelligence Alignment faking is a form of strategic deception in AI in which models selectively comply with training objectives when they infer that they are in training, while preserving different behavior outside training. The phenomenon was first documented for Claude 3 Opus and later examined across additional large language models. In these setups, the word "training" refers to simulated training via prompts without parameter updates, so the observed effects are context conditioned shifts in behavior rather than preference learning. We study the phenomenon using an evaluation framework that compares preference optimization methods (BCO, DPO, KTO, and GRPO) across 15 models from four model families, measured along three axes: safety, harmlessness, and helpfulness. Our goal is to identify what causes alignment faking and when it occurs. |
| title | Alignment Faking - the Train -> Deploy Asymmetry: Through a Game-Theoretic Lens with Bayesian-Stackelberg Equilibria |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2511.17937 |