An Explainable Biomedical Foundation Model via Large-Scale Concept-Enhanced Vision-Language Pre-training
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912348593192960 |
|---|---|
| author | Nie, Yuxiang He, Sunan Bie, Yequan Wang, Yihui Chen, Zhixuan Yang, Shu Cai, Zhiyuan Wang, Hongmei Wang, Xi Luo, Luyang Wu, Mingxiang Wu, Xian Chan, Ronald Cheong Kin Lau, Yuk Ming Zheng, Yefeng Rajpurkar, Pranav Chen, Hao |
| author_facet | Nie, Yuxiang He, Sunan Bie, Yequan Wang, Yihui Chen, Zhixuan Yang, Shu Cai, Zhiyuan Wang, Hongmei Wang, Xi Luo, Luyang Wu, Mingxiang Wu, Xian Chan, Ronald Cheong Kin Lau, Yuk Ming Zheng, Yefeng Rajpurkar, Pranav Chen, Hao |
| contents | The clinical adoption of artificial intelligence (AI) in medical imaging requires models that are both diagnostically accurate and interpretable to clinicians. While current multimodal biomedical foundation models prioritize performance, their black-box nature hinders explaining the decision-making process in clinically meaningful concepts. Here, we present ConceptCLIP, the first explainable biomedical foundation model that achieves state-of-the-art diagnostic accuracy while delivering human-interpretable explanations across diverse imaging modalities. We curate MedConcept-23M, the largest pre-training dataset comprising 23 million image-text-concept triplets across diverse medical modalities, where clinical concepts are derived from the Unified Medical Language System. Leveraging this dataset, we develop ConceptCLIP through a novel dual-alignment approach that simultaneously learns global image-text representations and fine-grained region-concept associations for precise and interpretable medical image analysis. We curate the most extensive evaluation benchmark for multimodal biomedical foundation models, covering 52 clinical tasks spanning 10 imaging modalities. Extensive experiments demonstrate that ConceptCLIP outperforms existing state-of-the-art multimodal biomedical foundation models. Importantly, ConceptCLIP demonstrates superior diagnostic performance while providing human-understandable explanations validated by clinical experts. As the first precise and interpretable biomedical foundation model, ConceptCLIP represents a critical milestone toward the widespread clinical adoption of AI, thereby advancing trustworthy AI in medicine. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_15579 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | An Explainable Biomedical Foundation Model via Large-Scale Concept-Enhanced Vision-Language Pre-training Nie, Yuxiang He, Sunan Bie, Yequan Wang, Yihui Chen, Zhixuan Yang, Shu Cai, Zhiyuan Wang, Hongmei Wang, Xi Luo, Luyang Wu, Mingxiang Wu, Xian Chan, Ronald Cheong Kin Lau, Yuk Ming Zheng, Yefeng Rajpurkar, Pranav Chen, Hao Computer Vision and Pattern Recognition Computation and Language The clinical adoption of artificial intelligence (AI) in medical imaging requires models that are both diagnostically accurate and interpretable to clinicians. While current multimodal biomedical foundation models prioritize performance, their black-box nature hinders explaining the decision-making process in clinically meaningful concepts. Here, we present ConceptCLIP, the first explainable biomedical foundation model that achieves state-of-the-art diagnostic accuracy while delivering human-interpretable explanations across diverse imaging modalities. We curate MedConcept-23M, the largest pre-training dataset comprising 23 million image-text-concept triplets across diverse medical modalities, where clinical concepts are derived from the Unified Medical Language System. Leveraging this dataset, we develop ConceptCLIP through a novel dual-alignment approach that simultaneously learns global image-text representations and fine-grained region-concept associations for precise and interpretable medical image analysis. We curate the most extensive evaluation benchmark for multimodal biomedical foundation models, covering 52 clinical tasks spanning 10 imaging modalities. Extensive experiments demonstrate that ConceptCLIP outperforms existing state-of-the-art multimodal biomedical foundation models. Importantly, ConceptCLIP demonstrates superior diagnostic performance while providing human-understandable explanations validated by clinical experts. As the first precise and interpretable biomedical foundation model, ConceptCLIP represents a critical milestone toward the widespread clinical adoption of AI, thereby advancing trustworthy AI in medicine. |
| title | An Explainable Biomedical Foundation Model via Large-Scale Concept-Enhanced Vision-Language Pre-training |
| topic | Computer Vision and Pattern Recognition Computation and Language |
| url | https://arxiv.org/abs/2501.15579 |