Benchmarking foundation models as feature extractors for weakly-supervised computational pathology
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909420281135104 |
|---|---|
| author | Neidlinger, Peter Nahhas, Omar S. M. El Muti, Hannah Sophie Lenz, Tim Hoffmeister, Michael Brenner, Hermann van Treeck, Marko Langer, Rupert Dislich, Bastian Behrens, Hans Michael Röcken, Christoph Foersch, Sebastian Truhn, Daniel Marra, Antonio Saldanha, Oliver Lester Kather, Jakob Nikolas |
| author_facet | Neidlinger, Peter Nahhas, Omar S. M. El Muti, Hannah Sophie Lenz, Tim Hoffmeister, Michael Brenner, Hermann van Treeck, Marko Langer, Rupert Dislich, Bastian Behrens, Hans Michael Röcken, Christoph Foersch, Sebastian Truhn, Daniel Marra, Antonio Saldanha, Oliver Lester Kather, Jakob Nikolas |
| contents | Advancements in artificial intelligence have driven the development of numerous pathology foundation models capable of extracting clinically relevant information. However, there is currently limited literature independently evaluating these foundation models on truly external cohorts and clinically-relevant tasks to uncover adjustments for future improvements. In this study, we benchmarked 19 histopathology foundation models on 13 patient cohorts with 6,818 patients and 9,528 slides from lung, colorectal, gastric, and breast cancers. The models were evaluated on weakly-supervised tasks related to biomarkers, morphological properties, and prognostic outcomes. We show that a vision-language foundation model, CONCH, yielded the highest performance when compared to vision-only foundation models, with Virchow2 as close second. The experiments reveal that foundation models trained on distinct cohorts learn complementary features to predict the same label, and can be fused to outperform the current state of the art. An ensemble combining CONCH and Virchow2 predictions outperformed individual models in 55% of tasks, leveraging their complementary strengths in classification scenarios. Moreover, our findings suggest that data diversity outweighs data volume for foundation models. Our work highlights actionable adjustments to improve pathology foundation models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_15823 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Benchmarking foundation models as feature extractors for weakly-supervised computational pathology Neidlinger, Peter Nahhas, Omar S. M. El Muti, Hannah Sophie Lenz, Tim Hoffmeister, Michael Brenner, Hermann van Treeck, Marko Langer, Rupert Dislich, Bastian Behrens, Hans Michael Röcken, Christoph Foersch, Sebastian Truhn, Daniel Marra, Antonio Saldanha, Oliver Lester Kather, Jakob Nikolas Image and Video Processing Computer Vision and Pattern Recognition Advancements in artificial intelligence have driven the development of numerous pathology foundation models capable of extracting clinically relevant information. However, there is currently limited literature independently evaluating these foundation models on truly external cohorts and clinically-relevant tasks to uncover adjustments for future improvements. In this study, we benchmarked 19 histopathology foundation models on 13 patient cohorts with 6,818 patients and 9,528 slides from lung, colorectal, gastric, and breast cancers. The models were evaluated on weakly-supervised tasks related to biomarkers, morphological properties, and prognostic outcomes. We show that a vision-language foundation model, CONCH, yielded the highest performance when compared to vision-only foundation models, with Virchow2 as close second. The experiments reveal that foundation models trained on distinct cohorts learn complementary features to predict the same label, and can be fused to outperform the current state of the art. An ensemble combining CONCH and Virchow2 predictions outperformed individual models in 55% of tasks, leveraging their complementary strengths in classification scenarios. Moreover, our findings suggest that data diversity outweighs data volume for foundation models. Our work highlights actionable adjustments to improve pathology foundation models. |
| title | Benchmarking foundation models as feature extractors for weakly-supervised computational pathology |
| topic | Image and Video Processing Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2408.15823 |