How Generative AI Empowers Attackers and Defenders Across the Trust & Safety Landscape
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910158059208704 |
|---|---|
| author | Kelley, Patrick Gage Rousso-Schindler, Steven Shelby, Renee Thomas, Kurt Woodruff, Allison |
| author_facet | Kelley, Patrick Gage Rousso-Schindler, Steven Shelby, Renee Thomas, Kurt Woodruff, Allison |
| contents | Generative AI (GenAI) is a powerful technology poised to reshape Trust & Safety. While misuse by attackers is a growing concern, its defensive capacity remains underexplored. This paper examines these effects through a qualitative study with 43 Trust & Safety experts across five domains: child safety, election integrity, hate and harassment, scams, and violent extremism. Our findings characterize a landscape in which GenAI empowers both attackers and defenders. GenAI dramatically increases the scale and speed of attacks, lowering the barrier to entry for creating harmful content, including sophisticated propaganda and deepfakes. Conversely, defenders envision leveraging GenAI to detect and mitigate harmful content at scale, conduct investigations, deploy persuasive counternarratives, improve moderator wellbeing, and offer user support. This work provides a strategic framework for understanding GenAI's impact on Trust & Safety and charts a path for its responsible use in creating safer online environments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_06033 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | How Generative AI Empowers Attackers and Defenders Across the Trust & Safety Landscape Kelley, Patrick Gage Rousso-Schindler, Steven Shelby, Renee Thomas, Kurt Woodruff, Allison Human-Computer Interaction Cryptography and Security Computers and Society Generative AI (GenAI) is a powerful technology poised to reshape Trust & Safety. While misuse by attackers is a growing concern, its defensive capacity remains underexplored. This paper examines these effects through a qualitative study with 43 Trust & Safety experts across five domains: child safety, election integrity, hate and harassment, scams, and violent extremism. Our findings characterize a landscape in which GenAI empowers both attackers and defenders. GenAI dramatically increases the scale and speed of attacks, lowering the barrier to entry for creating harmful content, including sophisticated propaganda and deepfakes. Conversely, defenders envision leveraging GenAI to detect and mitigate harmful content at scale, conduct investigations, deploy persuasive counternarratives, improve moderator wellbeing, and offer user support. This work provides a strategic framework for understanding GenAI's impact on Trust & Safety and charts a path for its responsible use in creating safer online environments. |
| title | How Generative AI Empowers Attackers and Defenders Across the Trust & Safety Landscape |
| topic | Human-Computer Interaction Cryptography and Security Computers and Society |
| url | https://arxiv.org/abs/2601.06033 |