Dealing with Annotator Disagreement in Hate Speech Classification
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913994567057408 |
|---|---|
| author | Dehghan, Somaiyeh Sen, Mehmet Umut Yanikoglu, Berrin |
| author_facet | Dehghan, Somaiyeh Sen, Mehmet Umut Yanikoglu, Berrin |
| contents | Hate speech detection is a crucial task, especially on social media, where harmful content can spread quickly. Implementing machine learning models to automatically identify and address hate speech is essential for mitigating its impact and preventing its proliferation. The first step in developing an effective hate speech detection model is to acquire a high-quality dataset for training. Labeled data is essential for most natural language processing tasks, but categorizing hate speech is difficult due to the diverse and often subjective nature of hate speech, which can lead to varying interpretations and disagreements among annotators. This paper examines strategies for addressing annotator disagreement, an issue that has been largely overlooked. In particular, we evaluate various automatic approaches for aggregating multiple annotations, in the context of hate speech classification in Turkish tweets. Our work highlights the importance of the problem and provides state-of-the-art benchmark results for the detection and understanding of hate speech in online discourse. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_08266 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Dealing with Annotator Disagreement in Hate Speech Classification Dehghan, Somaiyeh Sen, Mehmet Umut Yanikoglu, Berrin Computation and Language Artificial Intelligence Machine Learning I.2; I.2.7 Hate speech detection is a crucial task, especially on social media, where harmful content can spread quickly. Implementing machine learning models to automatically identify and address hate speech is essential for mitigating its impact and preventing its proliferation. The first step in developing an effective hate speech detection model is to acquire a high-quality dataset for training. Labeled data is essential for most natural language processing tasks, but categorizing hate speech is difficult due to the diverse and often subjective nature of hate speech, which can lead to varying interpretations and disagreements among annotators. This paper examines strategies for addressing annotator disagreement, an issue that has been largely overlooked. In particular, we evaluate various automatic approaches for aggregating multiple annotations, in the context of hate speech classification in Turkish tweets. Our work highlights the importance of the problem and provides state-of-the-art benchmark results for the detection and understanding of hate speech in online discourse. |
| title | Dealing with Annotator Disagreement in Hate Speech Classification |
| topic | Computation and Language Artificial Intelligence Machine Learning I.2; I.2.7 |
| url | https://arxiv.org/abs/2502.08266 |