ScamPilot: Simulating Conversations with LLMs to Protect Against Online Scams
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915762050957312 |
|---|---|
| author | Hoffman, Owen Peng, Kangze Kamal, Sajid You, Zehua Venkatagiri, Sukrit |
| author_facet | Hoffman, Owen Peng, Kangze Kamal, Sajid You, Zehua Venkatagiri, Sukrit |
| contents | Fraud continues to proliferate online, from phishing and ransomware to impersonation scams. Yet automated prevention approaches adapt slowly and may not reliably protect users from falling prey to new scams. To better combat online scams, we developed ScamPilot, a conversational interface that inoculates users against scams through simulation, dynamic interaction, and real-time feedback. ScamPilot simulates scams with two large language model-powered agents: a scammer and a target. Users must help the target defend against the scammer by providing real-time advice. Through a between-subjects study (N=150) with one control and three experimental conditions, we find that blending advice-giving with multiple choice questions significantly increased scam recognition (+8%) without decreasing wariness towards legitimate conversations. Users' response efficacy and change in self-efficacy was also 9% and 19% higher, respectively. Qualitatively, we find that users more frequently provided action-oriented advice over urging caution or providing emotional support. Overall, ScamPilot demonstrates the potential for inter-agent conversational user interfaces to augment learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_22426 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | ScamPilot: Simulating Conversations with LLMs to Protect Against Online Scams Hoffman, Owen Peng, Kangze Kamal, Sajid You, Zehua Venkatagiri, Sukrit Human-Computer Interaction Fraud continues to proliferate online, from phishing and ransomware to impersonation scams. Yet automated prevention approaches adapt slowly and may not reliably protect users from falling prey to new scams. To better combat online scams, we developed ScamPilot, a conversational interface that inoculates users against scams through simulation, dynamic interaction, and real-time feedback. ScamPilot simulates scams with two large language model-powered agents: a scammer and a target. Users must help the target defend against the scammer by providing real-time advice. Through a between-subjects study (N=150) with one control and three experimental conditions, we find that blending advice-giving with multiple choice questions significantly increased scam recognition (+8%) without decreasing wariness towards legitimate conversations. Users' response efficacy and change in self-efficacy was also 9% and 19% higher, respectively. Qualitatively, we find that users more frequently provided action-oriented advice over urging caution or providing emotional support. Overall, ScamPilot demonstrates the potential for inter-agent conversational user interfaces to augment learning. |
| title | ScamPilot: Simulating Conversations with LLMs to Protect Against Online Scams |
| topic | Human-Computer Interaction |
| url | https://arxiv.org/abs/2601.22426 |