Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2508.07308 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913982904795136 |
|---|---|
| author | Cosentino, Cristian Defilippo, Annamaria Dossena, Marco Irwin, Christopher Joubbi, Sara Liò, Pietro |
| author_facet | Cosentino, Cristian Defilippo, Annamaria Dossena, Marco Irwin, Christopher Joubbi, Sara Liò, Pietro |
| contents | HealthBranches is a novel benchmark dataset for medical Question-Answering (Q&A), specifically designed to evaluate complex reasoning in Large Language Models (LLMs). This dataset is generated through a semi-automated pipeline that transforms explicit decision pathways from medical source into realistic patient cases with associated questions and answers. Covering 4,063 case studies across 17 healthcare topics, each data point is based on clinically validated reasoning chains. HealthBranches supports both open-ended and multiple-choice question formats and uniquely includes the full reasoning path for each Q&A. Its structured design enables robust evaluation of LLMs' multi-step inference capabilities, including their performance in structured Retrieval-Augmented Generation (RAG) contexts. HealthBranches establishes a foundation for the development of more trustworthy, interpretable, and clinically reliable LLMs in high-stakes domains while also serving as a valuable resource for educational purposes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_07308 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | HealthBranches: Synthesizing Clinically-Grounded Question Answering Datasets via Decision Pathways Cosentino, Cristian Defilippo, Annamaria Dossena, Marco Irwin, Christopher Joubbi, Sara Liò, Pietro Computation and Language Artificial Intelligence Information Retrieval Machine Learning HealthBranches is a novel benchmark dataset for medical Question-Answering (Q&A), specifically designed to evaluate complex reasoning in Large Language Models (LLMs). This dataset is generated through a semi-automated pipeline that transforms explicit decision pathways from medical source into realistic patient cases with associated questions and answers. Covering 4,063 case studies across 17 healthcare topics, each data point is based on clinically validated reasoning chains. HealthBranches supports both open-ended and multiple-choice question formats and uniquely includes the full reasoning path for each Q&A. Its structured design enables robust evaluation of LLMs' multi-step inference capabilities, including their performance in structured Retrieval-Augmented Generation (RAG) contexts. HealthBranches establishes a foundation for the development of more trustworthy, interpretable, and clinically reliable LLMs in high-stakes domains while also serving as a valuable resource for educational purposes. |
| title | HealthBranches: Synthesizing Clinically-Grounded Question Answering Datasets via Decision Pathways |
| topic | Computation and Language Artificial Intelligence Information Retrieval Machine Learning |
| url | https://arxiv.org/abs/2508.07308 |