The Content Moderator's Dilemma: Removal of Toxic Content and Distortions to Online Discourse
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915829231124480 |
|---|---|
| author | Habibi, Mahyar Hovy, Dirk Schwarz, Carlo |
| author_facet | Habibi, Mahyar Hovy, Dirk Schwarz, Carlo |
| contents | There is an ongoing debate about how to moderate toxic speech on social media and the impact of content moderation on online discourse. This paper proposes and validates a methodology for measuring the content-moderation-induced distortions in online discourse using text embeddings from computational linguistics. Applying the method to a representative sample of 5 million US political Tweets, we find that removing toxic Tweets alters the semantic composition of content. This finding is consistent across different embedding models, toxicity metrics, and samples. Importantly, we demonstrate that these effects are not solely driven by toxic language but by the removal of topics often expressed in toxic form. We propose an alternative approach to content moderation that uses generative Large Language Models to rephrase toxic Tweets, preserving their salvageable content rather than removing them entirely. We show that this rephrasing strategy reduces toxicity while minimizing distortions in online content. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_16114 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | The Content Moderator's Dilemma: Removal of Toxic Content and Distortions to Online Discourse Habibi, Mahyar Hovy, Dirk Schwarz, Carlo Social and Information Networks There is an ongoing debate about how to moderate toxic speech on social media and the impact of content moderation on online discourse. This paper proposes and validates a methodology for measuring the content-moderation-induced distortions in online discourse using text embeddings from computational linguistics. Applying the method to a representative sample of 5 million US political Tweets, we find that removing toxic Tweets alters the semantic composition of content. This finding is consistent across different embedding models, toxicity metrics, and samples. Importantly, we demonstrate that these effects are not solely driven by toxic language but by the removal of topics often expressed in toxic form. We propose an alternative approach to content moderation that uses generative Large Language Models to rephrase toxic Tweets, preserving their salvageable content rather than removing them entirely. We show that this rephrasing strategy reduces toxicity while minimizing distortions in online content. |
| title | The Content Moderator's Dilemma: Removal of Toxic Content and Distortions to Online Discourse |
| topic | Social and Information Networks |
| url | https://arxiv.org/abs/2412.16114 |