Uncertainty Resolution in Misinformation Detection
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917557766717440 |
|---|---|
| author | Orlovskiy, Yury Thibault, Camille Imouza, Anne Godbout, Jean-François Rabbany, Reihaneh Pelrine, Kellin |
| author_facet | Orlovskiy, Yury Thibault, Camille Imouza, Anne Godbout, Jean-François Rabbany, Reihaneh Pelrine, Kellin |
| contents | Misinformation poses a variety of risks, such as undermining public trust and distorting factual discourse. Large Language Models (LLMs) like GPT-4 have been shown effective in mitigating misinformation, particularly in handling statements where enough context is provided. However, they struggle to assess ambiguous or context-deficient statements accurately. This work introduces a new method to resolve uncertainty in such statements. We propose a framework to categorize missing information and publish category labels for the LIAR-New dataset, which is adaptable to cross-domain content with missing information. We then leverage this framework to generate effective user queries for missing context. Compared to baselines, our method improves the rate at which generated questions are answerable by the user by 38 percentage points and classification performance by over 10 percentage points macro F1. Thus, this approach may provide a valuable component for future misinformation mitigation pipelines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_01197 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Uncertainty Resolution in Misinformation Detection Orlovskiy, Yury Thibault, Camille Imouza, Anne Godbout, Jean-François Rabbany, Reihaneh Pelrine, Kellin Computation and Language Artificial Intelligence Misinformation poses a variety of risks, such as undermining public trust and distorting factual discourse. Large Language Models (LLMs) like GPT-4 have been shown effective in mitigating misinformation, particularly in handling statements where enough context is provided. However, they struggle to assess ambiguous or context-deficient statements accurately. This work introduces a new method to resolve uncertainty in such statements. We propose a framework to categorize missing information and publish category labels for the LIAR-New dataset, which is adaptable to cross-domain content with missing information. We then leverage this framework to generate effective user queries for missing context. Compared to baselines, our method improves the rate at which generated questions are answerable by the user by 38 percentage points and classification performance by over 10 percentage points macro F1. Thus, this approach may provide a valuable component for future misinformation mitigation pipelines. |
| title | Uncertainty Resolution in Misinformation Detection |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2401.01197 |