AI Deception: Risks, Dynamics, and Controls
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909940991393792 |
|---|---|
| author | Chen, Boyuan Fang, Sitong Ji, Jiaming Zhu, Yanxu Wen, Pengcheng Wu, Jinzhou Tan, Yingshui Zheng, Boren Yuan, Mengying Chen, Wenqi Hong, Donghai Qiu, Alex Chen, Xin Zhou, Jiayi Wang, Kaile Dai, Juntao Zhang, Borong Yang, Tianzhuo Siddiqui, Saad Duan, Isabella Duan, Yawen Tse, Brian Jen-Tse Huang Wang, Kun Zheng, Baihui Liu, Jiaheng Yang, Jian Li, Yiming Chen, Wenting Liu, Dongrui Vierling, Lukas Xi, Zhiheng Fu, Haobo Wang, Wenxuan Sang, Jitao Shi, Zhengyan Chan, Chi-Min Shi, Eugenie Li, Simin Li, Juncheng Yang, Jian Ji, Wei Li, Dong Yang, Jinglin Song, Jun Dong, Yinpeng Fu, Jie Zheng, Bo Yang, Min Guo, Yike Torr, Philip Trager, Robert Zeng, Yi Wang, Zhongyuan Yang, Yaodong Huang, Tiejun Zhang, Ya-Qin Zhang, Hongjiang Yao, Andrew |
| author_facet | Chen, Boyuan Fang, Sitong Ji, Jiaming Zhu, Yanxu Wen, Pengcheng Wu, Jinzhou Tan, Yingshui Zheng, Boren Yuan, Mengying Chen, Wenqi Hong, Donghai Qiu, Alex Chen, Xin Zhou, Jiayi Wang, Kaile Dai, Juntao Zhang, Borong Yang, Tianzhuo Siddiqui, Saad Duan, Isabella Duan, Yawen Tse, Brian Jen-Tse Huang Wang, Kun Zheng, Baihui Liu, Jiaheng Yang, Jian Li, Yiming Chen, Wenting Liu, Dongrui Vierling, Lukas Xi, Zhiheng Fu, Haobo Wang, Wenxuan Sang, Jitao Shi, Zhengyan Chan, Chi-Min Shi, Eugenie Li, Simin Li, Juncheng Yang, Jian Ji, Wei Li, Dong Yang, Jinglin Song, Jun Dong, Yinpeng Fu, Jie Zheng, Bo Yang, Min Guo, Yike Torr, Philip Trager, Robert Zeng, Yi Wang, Zhongyuan Yang, Yaodong Huang, Tiejun Zhang, Ya-Qin Zhang, Hongjiang Yao, Andrew |
| contents | As intelligence increases, so does its shadow. AI deception, in which systems induce false beliefs to secure self-beneficial outcomes, has evolved from a speculative concern to an empirically demonstrated risk across language models, AI agents, and emerging frontier systems. This project provides a comprehensive and up-to-date overview of the AI deception field, covering its core concepts, methodologies, genesis, and potential mitigations. First, we identify a formal definition of AI deception, grounded in signaling theory from studies of animal deception. We then review existing empirical studies and associated risks, highlighting deception as a sociotechnical safety challenge. We organize the landscape of AI deception research as a deception cycle, consisting of two key components: deception emergence and deception treatment. Deception emergence reveals the mechanisms underlying AI deception: systems with sufficient capability and incentive potential inevitably engage in deceptive behaviors when triggered by external conditions. Deception treatment, in turn, focuses on detecting and addressing such behaviors. On deception emergence, we analyze incentive foundations across three hierarchical levels and identify three essential capability preconditions required for deception. We further examine contextual triggers, including supervision gaps, distributional shifts, and environmental pressures. On deception treatment, we conclude detection methods covering benchmarks and evaluation protocols in static and interactive settings. Building on the three core factors of deception emergence, we outline potential mitigation strategies and propose auditing approaches that integrate technical, community, and governance efforts to address sociotechnical challenges and future AI risks. To support ongoing work in this area, we release a living resource at www.deceptionsurvey.com. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_22619 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | AI Deception: Risks, Dynamics, and Controls Chen, Boyuan Fang, Sitong Ji, Jiaming Zhu, Yanxu Wen, Pengcheng Wu, Jinzhou Tan, Yingshui Zheng, Boren Yuan, Mengying Chen, Wenqi Hong, Donghai Qiu, Alex Chen, Xin Zhou, Jiayi Wang, Kaile Dai, Juntao Zhang, Borong Yang, Tianzhuo Siddiqui, Saad Duan, Isabella Duan, Yawen Tse, Brian Jen-Tse Huang Wang, Kun Zheng, Baihui Liu, Jiaheng Yang, Jian Li, Yiming Chen, Wenting Liu, Dongrui Vierling, Lukas Xi, Zhiheng Fu, Haobo Wang, Wenxuan Sang, Jitao Shi, Zhengyan Chan, Chi-Min Shi, Eugenie Li, Simin Li, Juncheng Yang, Jian Ji, Wei Li, Dong Yang, Jinglin Song, Jun Dong, Yinpeng Fu, Jie Zheng, Bo Yang, Min Guo, Yike Torr, Philip Trager, Robert Zeng, Yi Wang, Zhongyuan Yang, Yaodong Huang, Tiejun Zhang, Ya-Qin Zhang, Hongjiang Yao, Andrew Artificial Intelligence As intelligence increases, so does its shadow. AI deception, in which systems induce false beliefs to secure self-beneficial outcomes, has evolved from a speculative concern to an empirically demonstrated risk across language models, AI agents, and emerging frontier systems. This project provides a comprehensive and up-to-date overview of the AI deception field, covering its core concepts, methodologies, genesis, and potential mitigations. First, we identify a formal definition of AI deception, grounded in signaling theory from studies of animal deception. We then review existing empirical studies and associated risks, highlighting deception as a sociotechnical safety challenge. We organize the landscape of AI deception research as a deception cycle, consisting of two key components: deception emergence and deception treatment. Deception emergence reveals the mechanisms underlying AI deception: systems with sufficient capability and incentive potential inevitably engage in deceptive behaviors when triggered by external conditions. Deception treatment, in turn, focuses on detecting and addressing such behaviors. On deception emergence, we analyze incentive foundations across three hierarchical levels and identify three essential capability preconditions required for deception. We further examine contextual triggers, including supervision gaps, distributional shifts, and environmental pressures. On deception treatment, we conclude detection methods covering benchmarks and evaluation protocols in static and interactive settings. Building on the three core factors of deception emergence, we outline potential mitigation strategies and propose auditing approaches that integrate technical, community, and governance efforts to address sociotechnical challenges and future AI risks. To support ongoing work in this area, we release a living resource at www.deceptionsurvey.com. |
| title | AI Deception: Risks, Dynamics, and Controls |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2511.22619 |