Objection Overruled! Lay People can Distinguish Large Language Models from Lawyers, but still Favour Advice from an LLM
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916658541494272 |
|---|---|
| author | Schneiders, Eike Seabrooke, Tina Krook, Joshua Hyde, Richard Leesakul, Natalie Clos, Jeremie Fischer, Joel |
| author_facet | Schneiders, Eike Seabrooke, Tina Krook, Joshua Hyde, Richard Leesakul, Natalie Clos, Jeremie Fischer, Joel |
| contents | Large Language Models (LLMs) are seemingly infiltrating every domain, and the legal context is no exception. In this paper, we present the results of three experiments (total N = 288) that investigated lay people's willingness to act upon, and their ability to discriminate between, LLM- and lawyer-generated legal advice. In Experiment 1, participants judged their willingness to act on legal advice when the source of the advice was either known or unknown. When the advice source was unknown, participants indicated that they were significantly more willing to act on the LLM-generated advice. The result of the source unknown condition was replicated in Experiment 2. Intriguingly, despite participants indicating higher willingness to act on LLM-generated advice in Experiments 1 and 2, participants discriminated between the LLM- and lawyer-generated texts significantly above chance-level in Experiment 3. Lastly, we discuss potential explanations and risks of our findings, limitations and future work. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_07871 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Objection Overruled! Lay People can Distinguish Large Language Models from Lawyers, but still Favour Advice from an LLM Schneiders, Eike Seabrooke, Tina Krook, Joshua Hyde, Richard Leesakul, Natalie Clos, Jeremie Fischer, Joel Human-Computer Interaction Computers and Society Large Language Models (LLMs) are seemingly infiltrating every domain, and the legal context is no exception. In this paper, we present the results of three experiments (total N = 288) that investigated lay people's willingness to act upon, and their ability to discriminate between, LLM- and lawyer-generated legal advice. In Experiment 1, participants judged their willingness to act on legal advice when the source of the advice was either known or unknown. When the advice source was unknown, participants indicated that they were significantly more willing to act on the LLM-generated advice. The result of the source unknown condition was replicated in Experiment 2. Intriguingly, despite participants indicating higher willingness to act on LLM-generated advice in Experiments 1 and 2, participants discriminated between the LLM- and lawyer-generated texts significantly above chance-level in Experiment 3. Lastly, we discuss potential explanations and risks of our findings, limitations and future work. |
| title | Objection Overruled! Lay People can Distinguish Large Language Models from Lawyers, but still Favour Advice from an LLM |
| topic | Human-Computer Interaction Computers and Society |
| url | https://arxiv.org/abs/2409.07871 |