Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.21583 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914220355878912 |
|---|---|
| author | Zang, Zelin Gu, Wenyi Ma, Siqi Yang, Dan Shen, Yue Zhang, Zhu Fan, Guohui Ling, Wing-Kuen Yang, Fuji |
| author_facet | Zang, Zelin Gu, Wenyi Ma, Siqi Yang, Dan Shen, Yue Zhang, Zhu Fan, Guohui Ling, Wing-Kuen Yang, Fuji |
| contents | With the rapid growth of large language models (LLMs) and vision-language models (VLMs) in medicine, simply integrating clinical text and medical imaging does not guarantee reliable reasoning. Existing multimodal models often produce hallucinations or inconsistent chains of thought, limiting clinical trust. We propose a diagnostic framework built upon LLaVA that combines vision-language alignment with logic-regularized reasoning. The system includes an input encoder for text and images, a projection module for cross-modal alignment, a reasoning controller that decomposes diagnostic tasks into steps, and a logic tree generator that assembles stepwise premises into verifiable conclusions. Evaluations on MedXpertQA and other benchmarks show that our method improves diagnostic accuracy and yields more interpretable reasoning traces on multimodal tasks, while remaining competitive on text-only settings. These results suggest a promising step toward trustworthy multimodal medical AI. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_21583 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Medical Multimodal Diagnostic Framework Integrating Vision-Language Models and Logic Tree Reasoning Zang, Zelin Gu, Wenyi Ma, Siqi Yang, Dan Shen, Yue Zhang, Zhu Fan, Guohui Ling, Wing-Kuen Yang, Fuji Artificial Intelligence With the rapid growth of large language models (LLMs) and vision-language models (VLMs) in medicine, simply integrating clinical text and medical imaging does not guarantee reliable reasoning. Existing multimodal models often produce hallucinations or inconsistent chains of thought, limiting clinical trust. We propose a diagnostic framework built upon LLaVA that combines vision-language alignment with logic-regularized reasoning. The system includes an input encoder for text and images, a projection module for cross-modal alignment, a reasoning controller that decomposes diagnostic tasks into steps, and a logic tree generator that assembles stepwise premises into verifiable conclusions. Evaluations on MedXpertQA and other benchmarks show that our method improves diagnostic accuracy and yields more interpretable reasoning traces on multimodal tasks, while remaining competitive on text-only settings. These results suggest a promising step toward trustworthy multimodal medical AI. |
| title | A Medical Multimodal Diagnostic Framework Integrating Vision-Language Models and Logic Tree Reasoning |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2512.21583 |