Can LLMs Automate Fact-Checking Article Writing?
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910017055096832 |
|---|---|
| author | Sahnan, Dhruv Corney, David Larraz, Irene Zagni, Giovanni Miguez, Ruben Xie, Zhuohan Gurevych, Iryna Churchill, Elizabeth Chakraborty, Tanmoy Nakov, Preslav |
| author_facet | Sahnan, Dhruv Corney, David Larraz, Irene Zagni, Giovanni Miguez, Ruben Xie, Zhuohan Gurevych, Iryna Churchill, Elizabeth Chakraborty, Tanmoy Nakov, Preslav |
| contents | Automatic fact-checking aims to support professional fact-checkers by offering tools that can help speed up manual fact-checking. Yet, existing frameworks fail to address the key step of producing output suitable for broader dissemination to the general public: while human fact-checkers communicate their findings through fact-checking articles, automated systems typically produce little or no justification for their assessments. Here, we aim to bridge this gap. In particular, we argue for the need to extend the typical automatic fact-checking pipeline with automatic generation of full fact-checking articles. We first identify key desiderata for such articles through a series of interviews with experts from leading fact-checking organizations. We then develop QRAFT, an LLM-based agentic framework that mimics the writing workflow of human fact-checkers. Finally, we assess the practical usefulness of QRAFT through human evaluations with professional fact-checkers. Our evaluation shows that while QRAFT outperforms several previously proposed text-generation approaches, it lags considerably behind expert-written articles. We hope that our work will enable further research in this new and important direction. The code for our implementation is available at https://github.com/mbzuai-nlp/qraft.git. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_17684 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Can LLMs Automate Fact-Checking Article Writing? Sahnan, Dhruv Corney, David Larraz, Irene Zagni, Giovanni Miguez, Ruben Xie, Zhuohan Gurevych, Iryna Churchill, Elizabeth Chakraborty, Tanmoy Nakov, Preslav Computation and Language Artificial Intelligence Automatic fact-checking aims to support professional fact-checkers by offering tools that can help speed up manual fact-checking. Yet, existing frameworks fail to address the key step of producing output suitable for broader dissemination to the general public: while human fact-checkers communicate their findings through fact-checking articles, automated systems typically produce little or no justification for their assessments. Here, we aim to bridge this gap. In particular, we argue for the need to extend the typical automatic fact-checking pipeline with automatic generation of full fact-checking articles. We first identify key desiderata for such articles through a series of interviews with experts from leading fact-checking organizations. We then develop QRAFT, an LLM-based agentic framework that mimics the writing workflow of human fact-checkers. Finally, we assess the practical usefulness of QRAFT through human evaluations with professional fact-checkers. Our evaluation shows that while QRAFT outperforms several previously proposed text-generation approaches, it lags considerably behind expert-written articles. We hope that our work will enable further research in this new and important direction. The code for our implementation is available at https://github.com/mbzuai-nlp/qraft.git. |
| title | Can LLMs Automate Fact-Checking Article Writing? |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2503.17684 |