Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2405.11030 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910452650344448 |
|---|---|
| author | Wang, Xinyu Koneru, Sai Venkit, Pranav Narayanan Frischmann, Brett Rajtmajer, Sarah |
| author_facet | Wang, Xinyu Koneru, Sai Venkit, Pranav Narayanan Frischmann, Brett Rajtmajer, Sarah |
| contents | As social media has become a predominant mode of communication globally, the rise of abusive content threatens to undermine civil discourse. Recognizing the critical nature of this issue, a significant body of research has been dedicated to developing language models that can detect various types of online abuse, e.g., hate speech, cyberbullying. However, there exists a notable disconnect between platform policies, which often consider the author's intention as a criterion for content moderation, and the current capabilities of detection models, which typically lack efforts to capture intent. This paper examines the role of intent in content moderation systems. We review state of the art detection models and benchmark training datasets for online abuse to assess their awareness and ability to capture intent. We propose strategic changes to the design and development of automated detection and moderation systems to improve alignment with ethical and policy conceptualizations of abuse. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_11030 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | The Unappreciated Role of Intent in Algorithmic Moderation of Social Media Content Wang, Xinyu Koneru, Sai Venkit, Pranav Narayanan Frischmann, Brett Rajtmajer, Sarah Computation and Language As social media has become a predominant mode of communication globally, the rise of abusive content threatens to undermine civil discourse. Recognizing the critical nature of this issue, a significant body of research has been dedicated to developing language models that can detect various types of online abuse, e.g., hate speech, cyberbullying. However, there exists a notable disconnect between platform policies, which often consider the author's intention as a criterion for content moderation, and the current capabilities of detection models, which typically lack efforts to capture intent. This paper examines the role of intent in content moderation systems. We review state of the art detection models and benchmark training datasets for online abuse to assess their awareness and ability to capture intent. We propose strategic changes to the design and development of automated detection and moderation systems to improve alignment with ethical and policy conceptualizations of abuse. |
| title | The Unappreciated Role of Intent in Algorithmic Moderation of Social Media Content |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2405.11030 |