Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2405.20505 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913371453915136 |
|---|---|
| author | Yvinec, Edouard Kasser, Gabriel |
| author_facet | Yvinec, Edouard Kasser, Gabriel |
| contents | The wide acceptance of large language models (LLMs) has unlocked new applications and social risks. Popular countermeasures aim at detecting misinformation, usually involve domain specific models trained to recognize the relevance of any information. Instead of evaluating the validity of the information, we propose to investigate LLM generated text from the perspective of trust. In this study, we define trust as the ability to know if an input text was generated by a LLM or a human. To do so, we design SPOT, an efficient method, that classifies the source of any, standalone, text input based on originality score. This score is derived from the prediction of a given LLM to detect other LLMs. We empirically demonstrate the robustness of the method to the architecture, training data, evaluation data, task and compression of modern LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_20505 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | SPOT: Text Source Prediction from Originality Score Thresholding Yvinec, Edouard Kasser, Gabriel Computation and Language Machine Learning The wide acceptance of large language models (LLMs) has unlocked new applications and social risks. Popular countermeasures aim at detecting misinformation, usually involve domain specific models trained to recognize the relevance of any information. Instead of evaluating the validity of the information, we propose to investigate LLM generated text from the perspective of trust. In this study, we define trust as the ability to know if an input text was generated by a LLM or a human. To do so, we design SPOT, an efficient method, that classifies the source of any, standalone, text input based on originality score. This score is derived from the prediction of a given LLM to detect other LLMs. We empirically demonstrate the robustness of the method to the architecture, training data, evaluation data, task and compression of modern LLMs. |
| title | SPOT: Text Source Prediction from Originality Score Thresholding |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2405.20505 |