A Mechanism-Based Approach to Mitigating Harms from Persuasive Generative AI
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913326109294592 |
|---|---|
| author | El-Sayed, Seliem Akbulut, Canfer McCroskery, Amanda Keeling, Geoff Kenton, Zachary Jalan, Zaria Marchal, Nahema Manzini, Arianna Shevlane, Toby Vallor, Shannon Susser, Daniel Franklin, Matija Bridgers, Sophie Law, Harry Rahtz, Matthew Shanahan, Murray Tessler, Michael Henry Douillard, Arthur Everitt, Tom Brown, Sasha |
| author_facet | El-Sayed, Seliem Akbulut, Canfer McCroskery, Amanda Keeling, Geoff Kenton, Zachary Jalan, Zaria Marchal, Nahema Manzini, Arianna Shevlane, Toby Vallor, Shannon Susser, Daniel Franklin, Matija Bridgers, Sophie Law, Harry Rahtz, Matthew Shanahan, Murray Tessler, Michael Henry Douillard, Arthur Everitt, Tom Brown, Sasha |
| contents | Recent generative AI systems have demonstrated more advanced persuasive capabilities and are increasingly permeating areas of life where they can influence decision-making. Generative AI presents a new risk profile of persuasion due the opportunity for reciprocal exchange and prolonged interactions. This has led to growing concerns about harms from AI persuasion and how they can be mitigated, highlighting the need for a systematic study of AI persuasion. The current definitions of AI persuasion are unclear and related harms are insufficiently studied. Existing harm mitigation approaches prioritise harms from the outcome of persuasion over harms from the process of persuasion. In this paper, we lay the groundwork for the systematic study of AI persuasion. We first put forward definitions of persuasive generative AI. We distinguish between rationally persuasive generative AI, which relies on providing relevant facts, sound reasoning, or other forms of trustworthy evidence, and manipulative generative AI, which relies on taking advantage of cognitive biases and heuristics or misrepresenting information. We also put forward a map of harms from AI persuasion, including definitions and examples of economic, physical, environmental, psychological, sociocultural, political, privacy, and autonomy harm. We then introduce a map of mechanisms that contribute to harmful persuasion. Lastly, we provide an overview of approaches that can be used to mitigate against process harms of persuasion, including prompt engineering for manipulation classification and red teaming. Future work will operationalise these mitigations and study the interaction between different types of mechanisms of persuasion. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_15058 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Mechanism-Based Approach to Mitigating Harms from Persuasive Generative AI El-Sayed, Seliem Akbulut, Canfer McCroskery, Amanda Keeling, Geoff Kenton, Zachary Jalan, Zaria Marchal, Nahema Manzini, Arianna Shevlane, Toby Vallor, Shannon Susser, Daniel Franklin, Matija Bridgers, Sophie Law, Harry Rahtz, Matthew Shanahan, Murray Tessler, Michael Henry Douillard, Arthur Everitt, Tom Brown, Sasha Computers and Society Artificial Intelligence Recent generative AI systems have demonstrated more advanced persuasive capabilities and are increasingly permeating areas of life where they can influence decision-making. Generative AI presents a new risk profile of persuasion due the opportunity for reciprocal exchange and prolonged interactions. This has led to growing concerns about harms from AI persuasion and how they can be mitigated, highlighting the need for a systematic study of AI persuasion. The current definitions of AI persuasion are unclear and related harms are insufficiently studied. Existing harm mitigation approaches prioritise harms from the outcome of persuasion over harms from the process of persuasion. In this paper, we lay the groundwork for the systematic study of AI persuasion. We first put forward definitions of persuasive generative AI. We distinguish between rationally persuasive generative AI, which relies on providing relevant facts, sound reasoning, or other forms of trustworthy evidence, and manipulative generative AI, which relies on taking advantage of cognitive biases and heuristics or misrepresenting information. We also put forward a map of harms from AI persuasion, including definitions and examples of economic, physical, environmental, psychological, sociocultural, political, privacy, and autonomy harm. We then introduce a map of mechanisms that contribute to harmful persuasion. Lastly, we provide an overview of approaches that can be used to mitigate against process harms of persuasion, including prompt engineering for manipulation classification and red teaming. Future work will operationalise these mitigations and study the interaction between different types of mechanisms of persuasion. |
| title | A Mechanism-Based Approach to Mitigating Harms from Persuasive Generative AI |
| topic | Computers and Society Artificial Intelligence |
| url | https://arxiv.org/abs/2404.15058 |