Data Scaling Laws for Radiology Foundation Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914039817306112 |
|---|---|
| author | Ilse, Maximilian Sharma, Harshita Schwaighofer, Anton Bond-Taylor, Sam Pérez-García, Fernando Melnichenko, Olesya Sykes, Anne-Marie G. Horst, Kelly K. Khandelwal, Ashish Reynolds, Maxwell Wetscherek, Maria T. Codella, Noel C. F. Alvarez-Valle, Javier Panagiotis, Korfiatis Salvatelli, Valentina |
| author_facet | Ilse, Maximilian Sharma, Harshita Schwaighofer, Anton Bond-Taylor, Sam Pérez-García, Fernando Melnichenko, Olesya Sykes, Anne-Marie G. Horst, Kelly K. Khandelwal, Ashish Reynolds, Maxwell Wetscherek, Maria T. Codella, Noel C. F. Alvarez-Valle, Javier Panagiotis, Korfiatis Salvatelli, Valentina |
| contents | Foundation vision encoders such as CLIP and DINOv2, trained on web-scale data, exhibit strong transfer performance across tasks and datasets. However, medical imaging foundation models remain constrained by smaller datasets, limiting our understanding of how data scale and pretraining paradigms affect performance in this setting. In this work, we systematically study continual pretraining of two vision encoders, MedImageInsight (MI2) and RAD-DINO representing the two major encoder paradigms CLIP and DINOv2, on up to 3.5M chest x-rays from a single institution, holding compute and evaluation protocols constant. We evaluate on classification (radiology findings, lines and tubes), segmentation (lines and tubes), and radiology report generation. While prior work has primarily focused on tasks related to radiology findings, we include lines and tubes tasks to counterbalance this bias and evaluate a model's ability to extract features that preserve continuity along elongated structures. Our experiments show that MI2 scales more effectively for finding-related tasks, while RAD-DINO is stronger on tube-related tasks. Surprisingly, continually pretraining MI2 with both reports and structured labels using UniCL improves performance, underscoring the value of structured supervision at scale. We further show that for some tasks, as few as 30k in-domain samples are sufficient to surpass open-weights foundation models. These results highlight the utility of center-specific continual pretraining, enabling medical institutions to derive significant performance gains by utilizing in-domain data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_12818 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Data Scaling Laws for Radiology Foundation Models Ilse, Maximilian Sharma, Harshita Schwaighofer, Anton Bond-Taylor, Sam Pérez-García, Fernando Melnichenko, Olesya Sykes, Anne-Marie G. Horst, Kelly K. Khandelwal, Ashish Reynolds, Maxwell Wetscherek, Maria T. Codella, Noel C. F. Alvarez-Valle, Javier Panagiotis, Korfiatis Salvatelli, Valentina Computer Vision and Pattern Recognition Artificial Intelligence Foundation vision encoders such as CLIP and DINOv2, trained on web-scale data, exhibit strong transfer performance across tasks and datasets. However, medical imaging foundation models remain constrained by smaller datasets, limiting our understanding of how data scale and pretraining paradigms affect performance in this setting. In this work, we systematically study continual pretraining of two vision encoders, MedImageInsight (MI2) and RAD-DINO representing the two major encoder paradigms CLIP and DINOv2, on up to 3.5M chest x-rays from a single institution, holding compute and evaluation protocols constant. We evaluate on classification (radiology findings, lines and tubes), segmentation (lines and tubes), and radiology report generation. While prior work has primarily focused on tasks related to radiology findings, we include lines and tubes tasks to counterbalance this bias and evaluate a model's ability to extract features that preserve continuity along elongated structures. Our experiments show that MI2 scales more effectively for finding-related tasks, while RAD-DINO is stronger on tube-related tasks. Surprisingly, continually pretraining MI2 with both reports and structured labels using UniCL improves performance, underscoring the value of structured supervision at scale. We further show that for some tasks, as few as 30k in-domain samples are sufficient to surpass open-weights foundation models. These results highlight the utility of center-specific continual pretraining, enabling medical institutions to derive significant performance gains by utilizing in-domain data. |
| title | Data Scaling Laws for Radiology Foundation Models |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2509.12818 |