MedReason: Eliciting Factual Medical Reasoning Steps in LLMs via Knowledge Graphs
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916674597289984 |
|---|---|
| author | Wu, Juncheng Deng, Wenlong Li, Xingxuan Liu, Sheng Mi, Taomian Peng, Yifan Xu, Ziyang Liu, Yi Cho, Hyunjin Choi, Chang-In Cao, Yihan Ren, Hui Li, Xiang Li, Xiaoxiao Zhou, Yuyin |
| author_facet | Wu, Juncheng Deng, Wenlong Li, Xingxuan Liu, Sheng Mi, Taomian Peng, Yifan Xu, Ziyang Liu, Yi Cho, Hyunjin Choi, Chang-In Cao, Yihan Ren, Hui Li, Xiang Li, Xiaoxiao Zhou, Yuyin |
| contents | Medical tasks such as diagnosis and treatment planning require precise and complex reasoning, particularly in life-critical domains. Unlike mathematical reasoning, medical reasoning demands meticulous, verifiable thought processes to ensure reliability and accuracy. However, there is a notable lack of datasets that provide transparent, step-by-step reasoning to validate and enhance the medical reasoning ability of AI models. To bridge this gap, we introduce MedReason, a large-scale high-quality medical reasoning dataset designed to enable faithful and explainable medical problem-solving in large language models (LLMs). We utilize a structured medical knowledge graph (KG) to convert clinical QA pairs into logical chains of reasoning, or ``thinking paths'', which trace connections from question elements to answers via relevant KG entities. Each path is validated for consistency with clinical logic and evidence-based medicine. Our pipeline generates detailed reasoning for various medical questions from 7 medical datasets, resulting in a dataset of 32,682 question-answer pairs, each with detailed, step-by-step explanations. Experiments demonstrate that fine-tuning with our dataset consistently boosts medical problem-solving capabilities, achieving significant gains of up to 7.7% for DeepSeek-Ditill-8B. Our top-performing model, MedReason-8B, outperforms the Huatuo-o1-8B, a state-of-the-art medical reasoning model, by up to 4.2% on the clinical benchmark MedBullets. We also engage medical professionals from diverse specialties to assess our dataset's quality, ensuring MedReason offers accurate and coherent medical reasoning. Our data, models, and code is available at https://github.com/UCSC-VLAA/MedReason. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_00993 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MedReason: Eliciting Factual Medical Reasoning Steps in LLMs via Knowledge Graphs Wu, Juncheng Deng, Wenlong Li, Xingxuan Liu, Sheng Mi, Taomian Peng, Yifan Xu, Ziyang Liu, Yi Cho, Hyunjin Choi, Chang-In Cao, Yihan Ren, Hui Li, Xiang Li, Xiaoxiao Zhou, Yuyin Computation and Language Artificial Intelligence Medical tasks such as diagnosis and treatment planning require precise and complex reasoning, particularly in life-critical domains. Unlike mathematical reasoning, medical reasoning demands meticulous, verifiable thought processes to ensure reliability and accuracy. However, there is a notable lack of datasets that provide transparent, step-by-step reasoning to validate and enhance the medical reasoning ability of AI models. To bridge this gap, we introduce MedReason, a large-scale high-quality medical reasoning dataset designed to enable faithful and explainable medical problem-solving in large language models (LLMs). We utilize a structured medical knowledge graph (KG) to convert clinical QA pairs into logical chains of reasoning, or ``thinking paths'', which trace connections from question elements to answers via relevant KG entities. Each path is validated for consistency with clinical logic and evidence-based medicine. Our pipeline generates detailed reasoning for various medical questions from 7 medical datasets, resulting in a dataset of 32,682 question-answer pairs, each with detailed, step-by-step explanations. Experiments demonstrate that fine-tuning with our dataset consistently boosts medical problem-solving capabilities, achieving significant gains of up to 7.7% for DeepSeek-Ditill-8B. Our top-performing model, MedReason-8B, outperforms the Huatuo-o1-8B, a state-of-the-art medical reasoning model, by up to 4.2% on the clinical benchmark MedBullets. We also engage medical professionals from diverse specialties to assess our dataset's quality, ensuring MedReason offers accurate and coherent medical reasoning. Our data, models, and code is available at https://github.com/UCSC-VLAA/MedReason. |
| title | MedReason: Eliciting Factual Medical Reasoning Steps in LLMs via Knowledge Graphs |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2504.00993 |