Web Retrieval Agents for Evidence-Based Misinformation Detection
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916430813855744 |
|---|---|
| author | Tian, Jacob-Junqi Yu, Hao Orlovskiy, Yury Vergho, Tyler Rivera, Mauricio Goel, Mayank Yang, Zachary Godbout, Jean-Francois Rabbany, Reihaneh Pelrine, Kellin |
| author_facet | Tian, Jacob-Junqi Yu, Hao Orlovskiy, Yury Vergho, Tyler Rivera, Mauricio Goel, Mayank Yang, Zachary Godbout, Jean-Francois Rabbany, Reihaneh Pelrine, Kellin |
| contents | This paper develops an agent-based automated fact-checking approach for detecting misinformation. We demonstrate that combining a powerful LLM agent, which does not have access to the internet for searches, with an online web search agent yields better results than when each tool is used independently. Our approach is robust across multiple models, outperforming alternatives and increasing the macro F1 of misinformation detection by as much as 20 percent compared to LLMs without search. We also conduct extensive analyses on the sources our system leverages and their biases, decisions in the construction of the system like the search tool and the knowledge base, the type of evidence needed and its impact on the results, and other parts of the overall process. By combining strong performance with in-depth understanding, we hope to provide building blocks for future search-enabled misinformation mitigation systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_00009 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Web Retrieval Agents for Evidence-Based Misinformation Detection Tian, Jacob-Junqi Yu, Hao Orlovskiy, Yury Vergho, Tyler Rivera, Mauricio Goel, Mayank Yang, Zachary Godbout, Jean-Francois Rabbany, Reihaneh Pelrine, Kellin Information Retrieval Artificial Intelligence This paper develops an agent-based automated fact-checking approach for detecting misinformation. We demonstrate that combining a powerful LLM agent, which does not have access to the internet for searches, with an online web search agent yields better results than when each tool is used independently. Our approach is robust across multiple models, outperforming alternatives and increasing the macro F1 of misinformation detection by as much as 20 percent compared to LLMs without search. We also conduct extensive analyses on the sources our system leverages and their biases, decisions in the construction of the system like the search tool and the knowledge base, the type of evidence needed and its impact on the results, and other parts of the overall process. By combining strong performance with in-depth understanding, we hope to provide building blocks for future search-enabled misinformation mitigation systems. |
| title | Web Retrieval Agents for Evidence-Based Misinformation Detection |
| topic | Information Retrieval Artificial Intelligence |
| url | https://arxiv.org/abs/2409.00009 |