GPT-4 is judged more human than humans in displaced and inverted Turing tests
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909251652288512 |
|---|---|
| author | Rathi, Ishika Taylor, Sydney Bergen, Benjamin K. Jones, Cameron R. |
| author_facet | Rathi, Ishika Taylor, Sydney Bergen, Benjamin K. Jones, Cameron R. |
| contents | Everyday AI detection requires differentiating between people and AI in informal, online conversations. In many cases, people will not interact directly with AI systems but instead read conversations between AI systems and other people. We measured how well people and large language models can discriminate using two modified versions of the Turing test: inverted and displaced. GPT-3.5, GPT-4, and displaced human adjudicators judged whether an agent was human or AI on the basis of a Turing test transcript. We found that both AI and displaced human judges were less accurate than interactive interrogators, with below chance accuracy overall. Moreover, all three judged the best-performing GPT-4 witness to be human more often than human witnesses. This suggests that both humans and current LLMs struggle to distinguish between the two when they are not actively interrogating the person, underscoring an urgent need for more accurate tools to detect AI in conversations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_08853 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | GPT-4 is judged more human than humans in displaced and inverted Turing tests Rathi, Ishika Taylor, Sydney Bergen, Benjamin K. Jones, Cameron R. Human-Computer Interaction Computation and Language Everyday AI detection requires differentiating between people and AI in informal, online conversations. In many cases, people will not interact directly with AI systems but instead read conversations between AI systems and other people. We measured how well people and large language models can discriminate using two modified versions of the Turing test: inverted and displaced. GPT-3.5, GPT-4, and displaced human adjudicators judged whether an agent was human or AI on the basis of a Turing test transcript. We found that both AI and displaced human judges were less accurate than interactive interrogators, with below chance accuracy overall. Moreover, all three judged the best-performing GPT-4 witness to be human more often than human witnesses. This suggests that both humans and current LLMs struggle to distinguish between the two when they are not actively interrogating the person, underscoring an urgent need for more accurate tools to detect AI in conversations. |
| title | GPT-4 is judged more human than humans in displaced and inverted Turing tests |
| topic | Human-Computer Interaction Computation and Language |
| url | https://arxiv.org/abs/2407.08853 |