Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2402.01158 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910316222218240 |
|---|---|
| author | Wang, Rongsheng Chen, Haoming Zhou, Ruizhe Ma, Han Duan, Yaofei Kang, Yanlan Yang, Songhua Fan, Baoyu Tan, Tao |
| author_facet | Wang, Rongsheng Chen, Haoming Zhou, Ruizhe Ma, Han Duan, Yaofei Kang, Yanlan Yang, Songhua Fan, Baoyu Tan, Tao |
| contents | ChatGPT and other general large language models (LLMs) have achieved remarkable success, but they have also raised concerns about the misuse of AI-generated texts. Existing AI-generated text detection models, such as based on BERT and RoBERTa, are prone to in-domain over-fitting, leading to poor out-of-domain (OOD) detection performance. In this paper, we first collected Chinese text responses generated by human experts and 9 types of LLMs, for which to multiple domains questions, and further created a dataset that mixed human-written sentences and sentences polished by LLMs. We then proposed LLM-Detector, a novel method for both document-level and sentence-level text detection through Instruction Tuning of LLMs. Our method leverages the wealth of knowledge LLMs acquire during pre-training, enabling them to detect the text they generate. Instruction tuning aligns the model's responses with the user's expected text detection tasks. Experimental results show that previous methods struggle with sentence-level AI-generated text detection and OOD detection. In contrast, our proposed method not only significantly outperforms baseline methods in both sentence-level and document-level text detection but also demonstrates strong generalization capabilities. Furthermore, since LLM-Detector is trained based on open-source LLMs, it is easy to customize for deployment. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_01158 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | LLM-Detector: Improving AI-Generated Chinese Text Detection with Open-Source LLM Instruction Tuning Wang, Rongsheng Chen, Haoming Zhou, Ruizhe Ma, Han Duan, Yaofei Kang, Yanlan Yang, Songhua Fan, Baoyu Tan, Tao Computation and Language ChatGPT and other general large language models (LLMs) have achieved remarkable success, but they have also raised concerns about the misuse of AI-generated texts. Existing AI-generated text detection models, such as based on BERT and RoBERTa, are prone to in-domain over-fitting, leading to poor out-of-domain (OOD) detection performance. In this paper, we first collected Chinese text responses generated by human experts and 9 types of LLMs, for which to multiple domains questions, and further created a dataset that mixed human-written sentences and sentences polished by LLMs. We then proposed LLM-Detector, a novel method for both document-level and sentence-level text detection through Instruction Tuning of LLMs. Our method leverages the wealth of knowledge LLMs acquire during pre-training, enabling them to detect the text they generate. Instruction tuning aligns the model's responses with the user's expected text detection tasks. Experimental results show that previous methods struggle with sentence-level AI-generated text detection and OOD detection. In contrast, our proposed method not only significantly outperforms baseline methods in both sentence-level and document-level text detection but also demonstrates strong generalization capabilities. Furthermore, since LLM-Detector is trained based on open-source LLMs, it is easy to customize for deployment. |
| title | LLM-Detector: Improving AI-Generated Chinese Text Detection with Open-Source LLM Instruction Tuning |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2402.01158 |