How Large Language Models are Transforming Machine-Paraphrased Plagiarism
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2022
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917584629137408 |
|---|---|
| author | Wahle, Jan Philip Ruas, Terry Kirstein, Frederic Gipp, Bela |
| author_facet | Wahle, Jan Philip Ruas, Terry Kirstein, Frederic Gipp, Bela |
| contents | The recent success of large language models for text generation poses a severe threat to academic integrity, as plagiarists can generate realistic paraphrases indistinguishable from original work. However, the role of large autoregressive transformers in generating machine-paraphrased plagiarism and their detection is still developing in the literature. This work explores T5 and GPT-3 for machine-paraphrase generation on scientific articles from arXiv, student theses, and Wikipedia. We evaluate the detection performance of six automated solutions and one commercial plagiarism detection software and perform a human study with 105 participants regarding their detection performance and the quality of generated examples. Our results suggest that large models can rewrite text humans have difficulty identifying as machine-paraphrased (53% mean acc.). Human experts rate the quality of paraphrases generated by GPT-3 as high as original texts (clarity 4.0/5, fluency 4.2/5, coherence 3.8/5). The best-performing detection model (GPT-3) achieves a 66% F1-score in detecting paraphrases. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2210_03568 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | How Large Language Models are Transforming Machine-Paraphrased Plagiarism Wahle, Jan Philip Ruas, Terry Kirstein, Frederic Gipp, Bela Computation and Language Artificial Intelligence The recent success of large language models for text generation poses a severe threat to academic integrity, as plagiarists can generate realistic paraphrases indistinguishable from original work. However, the role of large autoregressive transformers in generating machine-paraphrased plagiarism and their detection is still developing in the literature. This work explores T5 and GPT-3 for machine-paraphrase generation on scientific articles from arXiv, student theses, and Wikipedia. We evaluate the detection performance of six automated solutions and one commercial plagiarism detection software and perform a human study with 105 participants regarding their detection performance and the quality of generated examples. Our results suggest that large models can rewrite text humans have difficulty identifying as machine-paraphrased (53% mean acc.). Human experts rate the quality of paraphrases generated by GPT-3 as high as original texts (clarity 4.0/5, fluency 4.2/5, coherence 3.8/5). The best-performing detection model (GPT-3) achieves a 66% F1-score in detecting paraphrases. |
| title | How Large Language Models are Transforming Machine-Paraphrased Plagiarism |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2210.03568 |