Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2410.15623 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913557169307648 |
|---|---|
| author | He, Jianfei Wang, Lilin Wang, Jiaying Liu, Zhenyu Na, Hongbin Wang, Zimu Wang, Wei Chen, Qi |
| author_facet | He, Jianfei Wang, Lilin Wang, Jiaying Liu, Zhenyu Na, Hongbin Wang, Zimu Wang, Wei Chen, Qi |
| contents | Identifying offensive language is essential for maintaining safety and sustainability in the social media era. Though large language models (LLMs) have demonstrated encouraging potential in social media analytics, they lack thorough evaluation when in offensive language detection, particularly in multilingual environments. We for the first time evaluate multilingual offensive language detection of LLMs in three languages: English, Spanish, and German with three LLMs, GPT-3.5, Flan-T5, and Mistral, in both monolingual and multilingual settings. We further examine the impact of different prompt languages and augmented translation data for the task in non-English contexts. Furthermore, we discuss the impact of the inherent bias in LLMs and the datasets in the mispredictions related to sensitive topics. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_15623 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Guardians of Discourse: Evaluating LLMs on Multilingual Offensive Language Detection He, Jianfei Wang, Lilin Wang, Jiaying Liu, Zhenyu Na, Hongbin Wang, Zimu Wang, Wei Chen, Qi Computation and Language Identifying offensive language is essential for maintaining safety and sustainability in the social media era. Though large language models (LLMs) have demonstrated encouraging potential in social media analytics, they lack thorough evaluation when in offensive language detection, particularly in multilingual environments. We for the first time evaluate multilingual offensive language detection of LLMs in three languages: English, Spanish, and German with three LLMs, GPT-3.5, Flan-T5, and Mistral, in both monolingual and multilingual settings. We further examine the impact of different prompt languages and augmented translation data for the task in non-English contexts. Furthermore, we discuss the impact of the inherent bias in LLMs and the datasets in the mispredictions related to sensitive topics. |
| title | Guardians of Discourse: Evaluating LLMs on Multilingual Offensive Language Detection |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2410.15623 |