Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.06186 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913867349622784 |
|---|---|
| author | Pronesti, Massimiliano Bettencourt-Silva, Joao Flanagan, Paul Pascale, Alessandra Redmond, Oisin Belz, Anya Hou, Yufang |
| author_facet | Pronesti, Massimiliano Bettencourt-Silva, Joao Flanagan, Paul Pascale, Alessandra Redmond, Oisin Belz, Anya Hou, Yufang |
| contents | Extracting scientific evidence from biomedical studies for clinical research questions (e.g., Does stem cell transplantation improve quality of life in patients with medically refractory Crohn's disease compared to placebo?) is a crucial step in synthesising biomedical evidence. In this paper, we focus on the task of document-level scientific evidence extraction for clinical questions with conflicting evidence. To support this task, we create a dataset called CochraneForest, leveraging forest plots from Cochrane systematic reviews. It comprises 202 annotated forest plots, associated clinical research questions, full texts of studies, and study-specific conclusions. Building on CochraneForest, we propose URCA (Uniform Retrieval Clustered Augmentation), a retrieval-augmented generation framework designed to tackle the unique challenges of evidence extraction. Our experiments show that URCA outperforms the best existing methods by up to 10.3% in F1 score on this task. However, the results also underscore the complexity of CochraneForest, establishing it as a challenging testbed for advancing automated evidence synthesis systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_06186 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Query-driven Document-level Scientific Evidence Extraction from Biomedical Studies Pronesti, Massimiliano Bettencourt-Silva, Joao Flanagan, Paul Pascale, Alessandra Redmond, Oisin Belz, Anya Hou, Yufang Computation and Language Artificial Intelligence Extracting scientific evidence from biomedical studies for clinical research questions (e.g., Does stem cell transplantation improve quality of life in patients with medically refractory Crohn's disease compared to placebo?) is a crucial step in synthesising biomedical evidence. In this paper, we focus on the task of document-level scientific evidence extraction for clinical questions with conflicting evidence. To support this task, we create a dataset called CochraneForest, leveraging forest plots from Cochrane systematic reviews. It comprises 202 annotated forest plots, associated clinical research questions, full texts of studies, and study-specific conclusions. Building on CochraneForest, we propose URCA (Uniform Retrieval Clustered Augmentation), a retrieval-augmented generation framework designed to tackle the unique challenges of evidence extraction. Our experiments show that URCA outperforms the best existing methods by up to 10.3% in F1 score on this task. However, the results also underscore the complexity of CochraneForest, establishing it as a challenging testbed for advancing automated evidence synthesis systems. |
| title | Query-driven Document-level Scientific Evidence Extraction from Biomedical Studies |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2505.06186 |