PLUTO: Pathology-Universal Transformer
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916244211367936 |
|---|---|
| author | Juyal, Dinkar Padigela, Harshith Shah, Chintan Shenker, Daniel Harguindeguy, Natalia Liu, Yi Martin, Blake Zhang, Yibo Nercessian, Michael Markey, Miles Finberg, Isaac Luu, Kelsey Borders, Daniel Javed, Syed Ashar Krause, Emma Biju, Raymond Sood, Aashish Ma, Allen Nyman, Jackson Shamshoian, John Chhor, Guillaume Sanghavi, Darpan Thibault, Marc Yu, Limin Najdawi, Fedaa Hipp, Jennifer A. Fahy, Darren Glass, Benjamin Walk, Eric Abel, John Pokkalla, Harsha Beck, Andrew H. Grullon, Sean |
| author_facet | Juyal, Dinkar Padigela, Harshith Shah, Chintan Shenker, Daniel Harguindeguy, Natalia Liu, Yi Martin, Blake Zhang, Yibo Nercessian, Michael Markey, Miles Finberg, Isaac Luu, Kelsey Borders, Daniel Javed, Syed Ashar Krause, Emma Biju, Raymond Sood, Aashish Ma, Allen Nyman, Jackson Shamshoian, John Chhor, Guillaume Sanghavi, Darpan Thibault, Marc Yu, Limin Najdawi, Fedaa Hipp, Jennifer A. Fahy, Darren Glass, Benjamin Walk, Eric Abel, John Pokkalla, Harsha Beck, Andrew H. Grullon, Sean |
| contents | Pathology is the study of microscopic inspection of tissue, and a pathology diagnosis is often the medical gold standard to diagnose disease. Pathology images provide a unique challenge for computer-vision-based analysis: a single pathology Whole Slide Image (WSI) is gigapixel-sized and often contains hundreds of thousands to millions of objects of interest across multiple resolutions. In this work, we propose PathoLogy Universal TransfOrmer (PLUTO): a light-weight pathology FM that is pre-trained on a diverse dataset of 195 million image tiles collected from multiple sites and extracts meaningful representations across multiple WSI scales that enable a large variety of downstream pathology tasks. In particular, we design task-specific adaptation heads that utilize PLUTO's output embeddings for tasks which span pathology scales ranging from subcellular to slide-scale, including instance segmentation, tile classification, and slide-level prediction. We compare PLUTO's performance to other state-of-the-art methods on a diverse set of external and internal benchmarks covering multiple biologically relevant tasks, tissue types, resolutions, stains, and scanners. We find that PLUTO matches or outperforms existing task-specific baselines and pathology-specific foundation models, some of which use orders-of-magnitude larger datasets and model sizes when compared to PLUTO. Our findings present a path towards a universal embedding to power pathology image analysis, and motivate further exploration around pathology foundation models in terms of data diversity, architectural improvements, sample efficiency, and practical deployability in real-world applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_07905 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | PLUTO: Pathology-Universal Transformer Juyal, Dinkar Padigela, Harshith Shah, Chintan Shenker, Daniel Harguindeguy, Natalia Liu, Yi Martin, Blake Zhang, Yibo Nercessian, Michael Markey, Miles Finberg, Isaac Luu, Kelsey Borders, Daniel Javed, Syed Ashar Krause, Emma Biju, Raymond Sood, Aashish Ma, Allen Nyman, Jackson Shamshoian, John Chhor, Guillaume Sanghavi, Darpan Thibault, Marc Yu, Limin Najdawi, Fedaa Hipp, Jennifer A. Fahy, Darren Glass, Benjamin Walk, Eric Abel, John Pokkalla, Harsha Beck, Andrew H. Grullon, Sean Image and Video Processing Computer Vision and Pattern Recognition Pathology is the study of microscopic inspection of tissue, and a pathology diagnosis is often the medical gold standard to diagnose disease. Pathology images provide a unique challenge for computer-vision-based analysis: a single pathology Whole Slide Image (WSI) is gigapixel-sized and often contains hundreds of thousands to millions of objects of interest across multiple resolutions. In this work, we propose PathoLogy Universal TransfOrmer (PLUTO): a light-weight pathology FM that is pre-trained on a diverse dataset of 195 million image tiles collected from multiple sites and extracts meaningful representations across multiple WSI scales that enable a large variety of downstream pathology tasks. In particular, we design task-specific adaptation heads that utilize PLUTO's output embeddings for tasks which span pathology scales ranging from subcellular to slide-scale, including instance segmentation, tile classification, and slide-level prediction. We compare PLUTO's performance to other state-of-the-art methods on a diverse set of external and internal benchmarks covering multiple biologically relevant tasks, tissue types, resolutions, stains, and scanners. We find that PLUTO matches or outperforms existing task-specific baselines and pathology-specific foundation models, some of which use orders-of-magnitude larger datasets and model sizes when compared to PLUTO. Our findings present a path towards a universal embedding to power pathology image analysis, and motivate further exploration around pathology foundation models in terms of data diversity, architectural improvements, sample efficiency, and practical deployability in real-world applications. |
| title | PLUTO: Pathology-Universal Transformer |
| topic | Image and Video Processing Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2405.07905 |