OPSD: an Offensive Persian Social media Dataset and its baseline evaluations
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929305841303552 |
|---|---|
| author | Safayani, Mehran Sartipi, Amir Ahmadi, Amir Hossein Jalali, Parniyan Mansouri, Amir Hossein Bisheh-Niasar, Mohammad Pourbahman, Zahra |
| author_facet | Safayani, Mehran Sartipi, Amir Ahmadi, Amir Hossein Jalali, Parniyan Mansouri, Amir Hossein Bisheh-Niasar, Mohammad Pourbahman, Zahra |
| contents | The proliferation of hate speech and offensive comments on social media has become increasingly prevalent due to user activities. Such comments can have detrimental effects on individuals' psychological well-being and social behavior. While numerous datasets in the English language exist in this domain, few equivalent resources are available for Persian language. To address this gap, this paper introduces two offensive datasets. The first dataset comprises annotations provided by domain experts, while the second consists of a large collection of unlabeled data obtained through web crawling for unsupervised learning purposes. To ensure the quality of the former dataset, a meticulous three-stage labeling process was conducted, and kappa measures were computed to assess inter-annotator agreement. Furthermore, experiments were performed on the dataset using state-of-the-art language models, both with and without employing masked language modeling techniques, as well as machine learning algorithms, in order to establish the baselines for the dataset using contemporary cutting-edge approaches. The obtained F1-scores for the three-class and two-class versions of the dataset were 76.9% and 89.9% for XLM-RoBERTa, respectively. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_05540 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | OPSD: an Offensive Persian Social media Dataset and its baseline evaluations Safayani, Mehran Sartipi, Amir Ahmadi, Amir Hossein Jalali, Parniyan Mansouri, Amir Hossein Bisheh-Niasar, Mohammad Pourbahman, Zahra Computation and Language Artificial Intelligence Machine Learning The proliferation of hate speech and offensive comments on social media has become increasingly prevalent due to user activities. Such comments can have detrimental effects on individuals' psychological well-being and social behavior. While numerous datasets in the English language exist in this domain, few equivalent resources are available for Persian language. To address this gap, this paper introduces two offensive datasets. The first dataset comprises annotations provided by domain experts, while the second consists of a large collection of unlabeled data obtained through web crawling for unsupervised learning purposes. To ensure the quality of the former dataset, a meticulous three-stage labeling process was conducted, and kappa measures were computed to assess inter-annotator agreement. Furthermore, experiments were performed on the dataset using state-of-the-art language models, both with and without employing masked language modeling techniques, as well as machine learning algorithms, in order to establish the baselines for the dataset using contemporary cutting-edge approaches. The obtained F1-scores for the three-class and two-class versions of the dataset were 76.9% and 89.9% for XLM-RoBERTa, respectively. |
| title | OPSD: an Offensive Persian Social media Dataset and its baseline evaluations |
| topic | Computation and Language Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2404.05540 |