Evaluating Vision Language Models (VLMs) for Radiology: A Comprehensive Analysis
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908332444352512 |
|---|---|
| author | Li, Frank Trivedi, Hari Khosravi, Bardia Dapamede, Theo Chavoshi, Mohammadreza Dere, Abdulhameed Isaac, Rohan Satya Mansuri, Aawez Newsome, Janice Purkayastha, Saptarshi Gichoya, Judy |
| author_facet | Li, Frank Trivedi, Hari Khosravi, Bardia Dapamede, Theo Chavoshi, Mohammadreza Dere, Abdulhameed Isaac, Rohan Satya Mansuri, Aawez Newsome, Janice Purkayastha, Saptarshi Gichoya, Judy |
| contents | Foundation models, trained on vast amounts of data using self-supervised techniques, have emerged as a promising frontier for advancing artificial intelligence (AI) applications in medicine. This study evaluates three different vision-language foundation models (RAD-DINO, CheXagent, and BiomedCLIP) on their ability to capture fine-grained imaging features for radiology tasks. The models were assessed across classification, segmentation, and regression tasks for pneumothorax and cardiomegaly on chest radiographs. Self-supervised RAD-DINO consistently excelled in segmentation tasks, while text-supervised CheXagent demonstrated superior classification performance. BiomedCLIP showed inconsistent performance across tasks. A custom segmentation model that integrates global and local features substantially improved performance for all foundation models, particularly for challenging pneumothorax segmentation. The findings highlight that pre-training methodology significantly influences model performance on specific downstream tasks. For fine-grained segmentation tasks, models trained without text supervision performed better, while text-supervised models offered advantages in classification and interpretability. These insights provide guidance for selecting foundation models based on specific clinical applications in radiology. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_16047 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Evaluating Vision Language Models (VLMs) for Radiology: A Comprehensive Analysis Li, Frank Trivedi, Hari Khosravi, Bardia Dapamede, Theo Chavoshi, Mohammadreza Dere, Abdulhameed Isaac, Rohan Satya Mansuri, Aawez Newsome, Janice Purkayastha, Saptarshi Gichoya, Judy Computer Vision and Pattern Recognition Artificial Intelligence Foundation models, trained on vast amounts of data using self-supervised techniques, have emerged as a promising frontier for advancing artificial intelligence (AI) applications in medicine. This study evaluates three different vision-language foundation models (RAD-DINO, CheXagent, and BiomedCLIP) on their ability to capture fine-grained imaging features for radiology tasks. The models were assessed across classification, segmentation, and regression tasks for pneumothorax and cardiomegaly on chest radiographs. Self-supervised RAD-DINO consistently excelled in segmentation tasks, while text-supervised CheXagent demonstrated superior classification performance. BiomedCLIP showed inconsistent performance across tasks. A custom segmentation model that integrates global and local features substantially improved performance for all foundation models, particularly for challenging pneumothorax segmentation. The findings highlight that pre-training methodology significantly influences model performance on specific downstream tasks. For fine-grained segmentation tasks, models trained without text supervision performed better, while text-supervised models offered advantages in classification and interpretability. These insights provide guidance for selecting foundation models based on specific clinical applications in radiology. |
| title | Evaluating Vision Language Models (VLMs) for Radiology: A Comprehensive Analysis |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2504.16047 |