Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2401.04481 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909066528292864 |
|---|---|
| author | Satapara, Shrey Mehta, Parth Ganguly, Debasis Modha, Sandip |
| author_facet | Satapara, Shrey Mehta, Parth Ganguly, Debasis Modha, Sandip |
| contents | The recent success in language generation capabilities of large language models (LLMs), such as GPT, Bard, Llama etc., can potentially lead to concerns about their possible misuse in inducing mass agitation and communal hatred via generating fake news and spreading misinformation. Traditional means of developing a misinformation ground-truth dataset does not scale well because of the extensive manual effort required to annotate the data. In this paper, we propose an LLM-based approach of creating silver-standard ground-truth datasets for identifying misinformation. Specifically speaking, given a trusted news article, our proposed approach involves prompting LLMs to automatically generate a summarised version of the original article. The prompts in our proposed approach act as a controlling mechanism to generate specific types of factual incorrectness in the generated summaries, e.g., incorrect quantities, false attributions etc. To investigate the usefulness of this dataset, we conduct a set of experiments where we train a range of supervised models for the task of misinformation detection. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_04481 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Fighting Fire with Fire: Adversarial Prompting to Generate a Misinformation Detection Dataset Satapara, Shrey Mehta, Parth Ganguly, Debasis Modha, Sandip Computation and Language Artificial Intelligence The recent success in language generation capabilities of large language models (LLMs), such as GPT, Bard, Llama etc., can potentially lead to concerns about their possible misuse in inducing mass agitation and communal hatred via generating fake news and spreading misinformation. Traditional means of developing a misinformation ground-truth dataset does not scale well because of the extensive manual effort required to annotate the data. In this paper, we propose an LLM-based approach of creating silver-standard ground-truth datasets for identifying misinformation. Specifically speaking, given a trusted news article, our proposed approach involves prompting LLMs to automatically generate a summarised version of the original article. The prompts in our proposed approach act as a controlling mechanism to generate specific types of factual incorrectness in the generated summaries, e.g., incorrect quantities, false attributions etc. To investigate the usefulness of this dataset, we conduct a set of experiments where we train a range of supervised models for the task of misinformation detection. |
| title | Fighting Fire with Fire: Adversarial Prompting to Generate a Misinformation Detection Dataset |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2401.04481 |