Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.24341 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912403888799744 |
|---|---|
| author | Yang, Shujian Cui, Shiyao Hu, Chuanrui Wang, Haicheng Zhang, Tianwei Huang, Minlie Lu, Jialiang Qiu, Han |
| author_facet | Yang, Shujian Cui, Shiyao Hu, Chuanrui Wang, Haicheng Zhang, Tianwei Huang, Minlie Lu, Jialiang Qiu, Han |
| contents | Detecting toxic content using language models is important but challenging. While large language models (LLMs) have demonstrated strong performance in understanding Chinese, recent studies show that simple character substitutions in toxic Chinese text can easily confuse the state-of-the-art (SOTA) LLMs. In this paper, we highlight the multimodal nature of Chinese language as a key challenge for deploying LLMs in toxic Chinese detection. First, we propose a taxonomy of 3 perturbation strategies and 8 specific approaches in toxic Chinese content. Then, we curate a dataset based on this taxonomy, and benchmark 9 SOTA LLMs (from both the US and China) to assess if they can detect perturbed toxic Chinese text. Additionally, we explore cost-effective enhancement solutions like in-context learning (ICL) and supervised fine-tuning (SFT). Our results reveal two important findings. (1) LLMs are less capable of detecting perturbed multimodal Chinese toxic contents. (2) ICL or SFT with a small number of perturbed examples may cause the LLMs "overcorrect'': misidentify many normal Chinese contents as toxic. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_24341 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Exploring Multimodal Challenges in Toxic Chinese Detection: Taxonomy, Benchmark, and Findings Yang, Shujian Cui, Shiyao Hu, Chuanrui Wang, Haicheng Zhang, Tianwei Huang, Minlie Lu, Jialiang Qiu, Han Computation and Language Artificial Intelligence Computers and Society Detecting toxic content using language models is important but challenging. While large language models (LLMs) have demonstrated strong performance in understanding Chinese, recent studies show that simple character substitutions in toxic Chinese text can easily confuse the state-of-the-art (SOTA) LLMs. In this paper, we highlight the multimodal nature of Chinese language as a key challenge for deploying LLMs in toxic Chinese detection. First, we propose a taxonomy of 3 perturbation strategies and 8 specific approaches in toxic Chinese content. Then, we curate a dataset based on this taxonomy, and benchmark 9 SOTA LLMs (from both the US and China) to assess if they can detect perturbed toxic Chinese text. Additionally, we explore cost-effective enhancement solutions like in-context learning (ICL) and supervised fine-tuning (SFT). Our results reveal two important findings. (1) LLMs are less capable of detecting perturbed multimodal Chinese toxic contents. (2) ICL or SFT with a small number of perturbed examples may cause the LLMs "overcorrect'': misidentify many normal Chinese contents as toxic. |
| title | Exploring Multimodal Challenges in Toxic Chinese Detection: Taxonomy, Benchmark, and Findings |
| topic | Computation and Language Artificial Intelligence Computers and Society |
| url | https://arxiv.org/abs/2505.24341 |