Pub-Guard-LLM: Detecting Retracted Biomedical Articles with Reliable Explanations
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915453616521216 |
|---|---|
| author | Chen, Lihu Fu, Shuojie Freedman, Gabriel Zor, Cemre Martin, Guy Kinross, James Vaghela, Uddhav Serban, Ovidiu Toni, Francesca |
| author_facet | Chen, Lihu Fu, Shuojie Freedman, Gabriel Zor, Cemre Martin, Guy Kinross, James Vaghela, Uddhav Serban, Ovidiu Toni, Francesca |
| contents | A significant and growing number of published scientific articles is found to involve fraudulent practices, posing a serious threat to the credibility and safety of research in fields such as medicine. We propose Pub-Guard-LLM, the first large language model-based system tailored to fraud detection of biomedical scientific articles. We provide three application modes for deploying Pub-Guard-LLM: vanilla reasoning, retrieval-augmented generation, and multi-agent debate. Each mode allows for textual explanations of predictions. To assess the performance of our system, we introduce an open-source benchmark, PubMed Retraction, comprising over 11K real-world biomedical articles, including metadata and retraction labels. We show that, across all modes, Pub-Guard-LLM consistently surpasses the performance of various baselines and provides more reliable explanations, namely explanations which are deemed more relevant and coherent than those generated by the baselines when evaluated by multiple assessment methods. By enhancing both detection performance and explainability in scientific fraud detection, Pub-Guard-LLM contributes to safeguarding research integrity with a novel, effective, open-source tool. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_15429 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Pub-Guard-LLM: Detecting Retracted Biomedical Articles with Reliable Explanations Chen, Lihu Fu, Shuojie Freedman, Gabriel Zor, Cemre Martin, Guy Kinross, James Vaghela, Uddhav Serban, Ovidiu Toni, Francesca Computation and Language A significant and growing number of published scientific articles is found to involve fraudulent practices, posing a serious threat to the credibility and safety of research in fields such as medicine. We propose Pub-Guard-LLM, the first large language model-based system tailored to fraud detection of biomedical scientific articles. We provide three application modes for deploying Pub-Guard-LLM: vanilla reasoning, retrieval-augmented generation, and multi-agent debate. Each mode allows for textual explanations of predictions. To assess the performance of our system, we introduce an open-source benchmark, PubMed Retraction, comprising over 11K real-world biomedical articles, including metadata and retraction labels. We show that, across all modes, Pub-Guard-LLM consistently surpasses the performance of various baselines and provides more reliable explanations, namely explanations which are deemed more relevant and coherent than those generated by the baselines when evaluated by multiple assessment methods. By enhancing both detection performance and explainability in scientific fraud detection, Pub-Guard-LLM contributes to safeguarding research integrity with a novel, effective, open-source tool. |
| title | Pub-Guard-LLM: Detecting Retracted Biomedical Articles with Reliable Explanations |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2502.15429 |