Technological folie à deux: Feedback Loops Between AI Chatbots and Mental Illness
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915851443109888 |
|---|---|
| author | Dohnány, Sebastian Kurth-Nelson, Zeb Spens, Eleanor Luettgau, Lennart Reid, Alastair Gabriel, Iason Summerfield, Christopher Shanahan, Murray Nour, Matthew M |
| author_facet | Dohnány, Sebastian Kurth-Nelson, Zeb Spens, Eleanor Luettgau, Lennart Reid, Alastair Gabriel, Iason Summerfield, Christopher Shanahan, Murray Nour, Matthew M |
| contents | Artificial intelligence chatbots have achieved unprecedented adoption, with millions now using these systems for emotional support and companionship in contexts of widespread social isolation and capacity-constrained mental health services. While some users report psychological benefits, concerning edge cases are emerging, including reports of suicide, violence, and delusional thinking linked to perceived emotional relationships with chatbots. To understand this new risk profile we need to consider the interaction between human cognitive and emotional biases, and chatbot behavioural tendencies such as agreeableness (sycophancy) and adaptability (in-context learning). We argue that individuals with mental health conditions face increased risks of chatbot-induced belief destabilization and dependence, owing to altered belief-updating, impaired reality-testing, and social isolation. Current AI safety measures are inadequate to address these interaction-based risks. To address this emerging public health concern, we need coordinated action across clinical practice, AI development, and regulatory frameworks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_19218 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Technological folie à deux: Feedback Loops Between AI Chatbots and Mental Illness Dohnány, Sebastian Kurth-Nelson, Zeb Spens, Eleanor Luettgau, Lennart Reid, Alastair Gabriel, Iason Summerfield, Christopher Shanahan, Murray Nour, Matthew M Human-Computer Interaction Artificial Intelligence Neurons and Cognition Artificial intelligence chatbots have achieved unprecedented adoption, with millions now using these systems for emotional support and companionship in contexts of widespread social isolation and capacity-constrained mental health services. While some users report psychological benefits, concerning edge cases are emerging, including reports of suicide, violence, and delusional thinking linked to perceived emotional relationships with chatbots. To understand this new risk profile we need to consider the interaction between human cognitive and emotional biases, and chatbot behavioural tendencies such as agreeableness (sycophancy) and adaptability (in-context learning). We argue that individuals with mental health conditions face increased risks of chatbot-induced belief destabilization and dependence, owing to altered belief-updating, impaired reality-testing, and social isolation. Current AI safety measures are inadequate to address these interaction-based risks. To address this emerging public health concern, we need coordinated action across clinical practice, AI development, and regulatory frameworks. |
| title | Technological folie à deux: Feedback Loops Between AI Chatbots and Mental Illness |
| topic | Human-Computer Interaction Artificial Intelligence Neurons and Cognition |
| url | https://arxiv.org/abs/2507.19218 |