Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.22338 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912299045879808 |
|---|---|
| author | Malviya, Shrikant Arnau-González, Pablo Arevalillo-Herráez, Miguel Katsigiannis, Stamos |
| author_facet | Malviya, Shrikant Arnau-González, Pablo Arevalillo-Herráez, Miguel Katsigiannis, Stamos |
| contents | The rapid advancement of large language models (LLMs) has introduced new challenges in distinguishing human-written text from AI-generated content. In this work, we explored a pipelined approach for AI-generated text detection that includes a feature extraction step (i.e. prompt-based rewriting features inspired by RAIDAR and content-based features derived from the NELA toolkit) followed by a classification module. Comprehensive experiments were conducted on the Defactify4.0 dataset, evaluating two tasks: binary classification to differentiate human-written and AI-generated text, and multi-class classification to identify the specific generative model used to generate the input text. Our findings reveal that NELA features significantly outperform RAIDAR features in both tasks, demonstrating their ability to capture nuanced linguistic, stylistic, and content-based differences. Combining RAIDAR and NELA features provided minimal improvement, highlighting the redundancy introduced by less discriminative features. Among the classifiers tested, XGBoost emerged as the most effective, leveraging the rich feature sets to achieve high accuracy and generalisation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_22338 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SKDU at De-Factify 4.0: Natural Language Features for AI-Generated Text-Detection Malviya, Shrikant Arnau-González, Pablo Arevalillo-Herráez, Miguel Katsigiannis, Stamos Computation and Language The rapid advancement of large language models (LLMs) has introduced new challenges in distinguishing human-written text from AI-generated content. In this work, we explored a pipelined approach for AI-generated text detection that includes a feature extraction step (i.e. prompt-based rewriting features inspired by RAIDAR and content-based features derived from the NELA toolkit) followed by a classification module. Comprehensive experiments were conducted on the Defactify4.0 dataset, evaluating two tasks: binary classification to differentiate human-written and AI-generated text, and multi-class classification to identify the specific generative model used to generate the input text. Our findings reveal that NELA features significantly outperform RAIDAR features in both tasks, demonstrating their ability to capture nuanced linguistic, stylistic, and content-based differences. Combining RAIDAR and NELA features provided minimal improvement, highlighting the redundancy introduced by less discriminative features. Among the classifiers tested, XGBoost emerged as the most effective, leveraging the rich feature sets to achieve high accuracy and generalisation. |
| title | SKDU at De-Factify 4.0: Natural Language Features for AI-Generated Text-Detection |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2503.22338 |