Toward Verifiable Misinformation Detection: A Multi-Tool LLM Agent Framework

Fuente: arXiv
Salvato in:
Dettagli Bibliografici
Autori principali: Cui, Zikun, Huang, Tianyi, Chiang, Chia-En, Du, Cuiqianhe
Natura: Preprint
Pubblicazione: 2025
Soggetti:
Accesso online:
Tags: Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
_version_ 1866909723068989440
author Cui, Zikun
Huang, Tianyi
Chiang, Chia-En
Du, Cuiqianhe
author_facet Cui, Zikun
Huang, Tianyi
Chiang, Chia-En
Du, Cuiqianhe
contents With the proliferation of Large Language Models (LLMs), the detection of misinformation has become increasingly important and complex. This research proposes an innovative verifiable misinformation detection LLM agent that goes beyond traditional true/false binary judgments. The agent actively verifies claims through dynamic interaction with diverse web sources, assesses information source credibility, synthesizes evidence, and provides a complete verifiable reasoning process. Our designed agent architecture includes three core tools: precise web search tool, source credibility assessment tool and numerical claim verification tool. These tools enable the agent to execute multi-step verification strategies, maintain evidence logs, and form comprehensive assessment conclusions. We evaluate using standard misinformation datasets such as FakeNewsNet, comparing with traditional machine learning models and LLMs. Evaluation metrics include standard classification metrics, quality assessment of reasoning processes, and robustness testing against rewritten content. Experimental results show that our agent outperforms baseline methods in misinformation detection accuracy, reasoning transparency, and resistance to information rewriting, providing a new paradigm for trustworthy AI-assisted fact-checking.
format Preprint
id arxiv_https___arxiv_org_abs_2508_03092
institution arXiv
publishDate 2025
record_format arxiv
spellingShingle Toward Verifiable Misinformation Detection: A Multi-Tool LLM Agent Framework
Cui, Zikun
Huang, Tianyi
Chiang, Chia-En
Du, Cuiqianhe
Artificial Intelligence
Computation and Language
With the proliferation of Large Language Models (LLMs), the detection of misinformation has become increasingly important and complex. This research proposes an innovative verifiable misinformation detection LLM agent that goes beyond traditional true/false binary judgments. The agent actively verifies claims through dynamic interaction with diverse web sources, assesses information source credibility, synthesizes evidence, and provides a complete verifiable reasoning process. Our designed agent architecture includes three core tools: precise web search tool, source credibility assessment tool and numerical claim verification tool. These tools enable the agent to execute multi-step verification strategies, maintain evidence logs, and form comprehensive assessment conclusions. We evaluate using standard misinformation datasets such as FakeNewsNet, comparing with traditional machine learning models and LLMs. Evaluation metrics include standard classification metrics, quality assessment of reasoning processes, and robustness testing against rewritten content. Experimental results show that our agent outperforms baseline methods in misinformation detection accuracy, reasoning transparency, and resistance to information rewriting, providing a new paradigm for trustworthy AI-assisted fact-checking.
title Toward Verifiable Misinformation Detection: A Multi-Tool LLM Agent Framework
topic Artificial Intelligence
Computation and Language
url https://arxiv.org/abs/2508.03092