Building a Silver-Standard Dataset from NICE Guidelines for Clinical LLMs
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909882707345408 |
|---|---|
| author | Ding, Qing Zhang, Eric Hua Qing Jozsa, Felix Ive, Julia |
| author_facet | Ding, Qing Zhang, Eric Hua Qing Jozsa, Felix Ive, Julia |
| contents | Large language models (LLMs) are increasingly used in healthcare, yet standardised benchmarks for evaluating guideline-based clinical reasoning are missing. This study introduces a validated dataset derived from publicly available guidelines across multiple diagnoses. The dataset was created with the help of GPT and contains realistic patient scenarios, as well as clinical questions. We benchmark a range of recent popular LLMs to showcase the validity of our dataset. The framework supports systematic evaluation of LLMs' clinical utility and guideline adherence. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_01053 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Building a Silver-Standard Dataset from NICE Guidelines for Clinical LLMs Ding, Qing Zhang, Eric Hua Qing Jozsa, Felix Ive, Julia Computation and Language Large language models (LLMs) are increasingly used in healthcare, yet standardised benchmarks for evaluating guideline-based clinical reasoning are missing. This study introduces a validated dataset derived from publicly available guidelines across multiple diagnoses. The dataset was created with the help of GPT and contains realistic patient scenarios, as well as clinical questions. We benchmark a range of recent popular LLMs to showcase the validity of our dataset. The framework supports systematic evaluation of LLMs' clinical utility and guideline adherence. |
| title | Building a Silver-Standard Dataset from NICE Guidelines for Clinical LLMs |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2511.01053 |