Measuring AI "Slop" in Text
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912844900990976 |
|---|---|
| author | Shaib, Chantal Chakrabarty, Tuhin Garcia-Olano, Diego Wallace, Byron C. |
| author_facet | Shaib, Chantal Chakrabarty, Tuhin Garcia-Olano, Diego Wallace, Byron C. |
| contents | AI "slop" is an increasingly popular term used to describe low-quality AI-generated text, but there is currently no agreed upon definition of this term nor a means to measure its occurrence. In this work, we develop a taxonomy of "slop" through interviews with experts in NLP, writing, and philosophy, and propose a set of interpretable dimensions for its assessment in text. Through span-level annotation, we find that binary "slop" judgments are (somewhat) subjective, but such determinations nonetheless correlate with latent dimensions such as coherence and relevance. Our framework can be used to evaluate AI-generated text in both detection and binary preference tasks, potentially offering new insights into the linguistic and stylistic factors that contribute to quality judgments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_19163 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Measuring AI "Slop" in Text Shaib, Chantal Chakrabarty, Tuhin Garcia-Olano, Diego Wallace, Byron C. Computation and Language AI "slop" is an increasingly popular term used to describe low-quality AI-generated text, but there is currently no agreed upon definition of this term nor a means to measure its occurrence. In this work, we develop a taxonomy of "slop" through interviews with experts in NLP, writing, and philosophy, and propose a set of interpretable dimensions for its assessment in text. Through span-level annotation, we find that binary "slop" judgments are (somewhat) subjective, but such determinations nonetheless correlate with latent dimensions such as coherence and relevance. Our framework can be used to evaluate AI-generated text in both detection and binary preference tasks, potentially offering new insights into the linguistic and stylistic factors that contribute to quality judgments. |
| title | Measuring AI "Slop" in Text |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2509.19163 |