Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2504.16604 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912514140274688 |
|---|---|
| author | Lisker, Mareike Gottschalk, Christina Mihaljević, Helena |
| author_facet | Lisker, Mareike Gottschalk, Christina Mihaljević, Helena |
| contents | Counterspeech is a key strategy against harmful online content, but scaling expert-driven efforts is challenging. Large Language Models (LLMs) present a potential solution, though their use in countering conspiracy theories is under-researched. Unlike for hate speech, no datasets exist that pair conspiracy theory comments with expert-crafted counterspeech. We address this gap by evaluating the ability of GPT-4o, Llama 3, and Mistral to effectively apply counterspeech strategies derived from psychological research provided through structured prompts. Our results show that the models often generate generic, repetitive, or superficial results. Additionally, they over-acknowledge fear and frequently hallucinate facts, sources, or figures, making their prompt-based use in practical applications problematic. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_16604 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Debunking with Dialogue? Exploring AI-Generated Counterspeech to Challenge Conspiracy Theories Lisker, Mareike Gottschalk, Christina Mihaljević, Helena Computation and Language Artificial Intelligence Social and Information Networks I.2.7 Counterspeech is a key strategy against harmful online content, but scaling expert-driven efforts is challenging. Large Language Models (LLMs) present a potential solution, though their use in countering conspiracy theories is under-researched. Unlike for hate speech, no datasets exist that pair conspiracy theory comments with expert-crafted counterspeech. We address this gap by evaluating the ability of GPT-4o, Llama 3, and Mistral to effectively apply counterspeech strategies derived from psychological research provided through structured prompts. Our results show that the models often generate generic, repetitive, or superficial results. Additionally, they over-acknowledge fear and frequently hallucinate facts, sources, or figures, making their prompt-based use in practical applications problematic. |
| title | Debunking with Dialogue? Exploring AI-Generated Counterspeech to Challenge Conspiracy Theories |
| topic | Computation and Language Artificial Intelligence Social and Information Networks I.2.7 |
| url | https://arxiv.org/abs/2504.16604 |