Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2605.03544 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914530698723328 |
|---|---|
| author | Lems, Carlijn Moonemans, Sander Klubíčková, Natálie Brattoli, Biagio Lee, Taebum Kim, Seokhwi Vilaplana, Veronica Pons, Laura Hochman, Sapir Suárez-Franck, Mauricio Eduardo Fernandez, Pedro Luis Drachneris, Julius Petroska, Donatas Augulis, Renaldas Laurinavicius, Arvydas Oliveira, Domingos Montezuma, Diana Bouwmeester, Anouk B. van Midden, Dominique Vos, Anne-Marie Vos, Shoko van Ipenburg, Jolique Balkenhol, Maschenka Winkler, Koen Nagtegaal, Iris Hebeda, Konnie Flucke, Uta Grünberg, Katrien Skopal, Josef Chohan, Brinder S. Temprana-Salvador, Jordi Munari, Enrico Cima, Luca Querzoli, Giulia Belisario, Yosamin Gonzalez Faber, Jaeike W. van Leenders, Geert J. L. H. von der Thüsen, Jan H. Brosens, Lodewijk A. A. de Krijger, Ronald R. Wesseling, Pieter Florquin, Sandrine Maniewski, Mateusz Kowalewski, Adam Barna, Robert Tiniakos, Dina Gros, Joan Lop Donders, Rogier Maurits, Jake S. F. Lu, Ming Yang Chen, Chengkuan Mahmood, Faisal van der Laak, Jeroen Khalili, Nadieh Meeuwsen, Frédérique Ciompi, Francesco |
| author_facet | Lems, Carlijn Moonemans, Sander Klubíčková, Natálie Brattoli, Biagio Lee, Taebum Kim, Seokhwi Vilaplana, Veronica Pons, Laura Hochman, Sapir Suárez-Franck, Mauricio Eduardo Fernandez, Pedro Luis Drachneris, Julius Petroska, Donatas Augulis, Renaldas Laurinavicius, Arvydas Oliveira, Domingos Montezuma, Diana Bouwmeester, Anouk B. van Midden, Dominique Vos, Anne-Marie Vos, Shoko van Ipenburg, Jolique Balkenhol, Maschenka Winkler, Koen Nagtegaal, Iris Hebeda, Konnie Flucke, Uta Grünberg, Katrien Skopal, Josef Chohan, Brinder S. Temprana-Salvador, Jordi Munari, Enrico Cima, Luca Querzoli, Giulia Belisario, Yosamin Gonzalez Faber, Jaeike W. van Leenders, Geert J. L. H. von der Thüsen, Jan H. Brosens, Lodewijk A. A. de Krijger, Ronald R. Wesseling, Pieter Florquin, Sandrine Maniewski, Mateusz Kowalewski, Adam Barna, Robert Tiniakos, Dina Gros, Joan Lop Donders, Rogier Maurits, Jake S. F. Lu, Ming Yang Chen, Chengkuan Mahmood, Faisal van der Laak, Jeroen Khalili, Nadieh Meeuwsen, Frédérique Ciompi, Francesco |
| contents | Foundation models with visual question answering capabilities for digital pathology are emerging. Such unprecedented technology requires independent benchmarking to assess its potential in assisting pathologists in routine diagnostics. We created DALPHIN, the first multicentric open benchmark for pathology AI copilots, comprising 1236 images from 300 cases, spanning 130 rare to common diagnoses, 6 countries, and 14 subspecialties. The DALPHIN design and dataset are introduced alongside a human performance benchmark of 31 pathologists from 10 countries with varying expertise. We report results for two general-purpose (GPT-5, Gemini 2.5 Pro) and one pathology-specific copilot (PathChat+) for sequential and independent answer generation. We observed no statistically significant difference from expert-level performance in four of six tasks for PathChat, 2/6 tasks for Gemini, and 1/6 tasks for GPT. DALPHIN is publicly released with sequestered, indirectly accessible ground truth to foster robust and enduring benchmarking. Data, methods, and the evaluation platform are accessible through dalphin.grand-challenge.org. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_03544 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | DALPHIN: Benchmarking Digital Pathology AI Copilots Against Pathologists on an Open Multicentric Dataset Lems, Carlijn Moonemans, Sander Klubíčková, Natálie Brattoli, Biagio Lee, Taebum Kim, Seokhwi Vilaplana, Veronica Pons, Laura Hochman, Sapir Suárez-Franck, Mauricio Eduardo Fernandez, Pedro Luis Drachneris, Julius Petroska, Donatas Augulis, Renaldas Laurinavicius, Arvydas Oliveira, Domingos Montezuma, Diana Bouwmeester, Anouk B. van Midden, Dominique Vos, Anne-Marie Vos, Shoko van Ipenburg, Jolique Balkenhol, Maschenka Winkler, Koen Nagtegaal, Iris Hebeda, Konnie Flucke, Uta Grünberg, Katrien Skopal, Josef Chohan, Brinder S. Temprana-Salvador, Jordi Munari, Enrico Cima, Luca Querzoli, Giulia Belisario, Yosamin Gonzalez Faber, Jaeike W. van Leenders, Geert J. L. H. von der Thüsen, Jan H. Brosens, Lodewijk A. A. de Krijger, Ronald R. Wesseling, Pieter Florquin, Sandrine Maniewski, Mateusz Kowalewski, Adam Barna, Robert Tiniakos, Dina Gros, Joan Lop Donders, Rogier Maurits, Jake S. F. Lu, Ming Yang Chen, Chengkuan Mahmood, Faisal van der Laak, Jeroen Khalili, Nadieh Meeuwsen, Frédérique Ciompi, Francesco Computer Vision and Pattern Recognition Artificial Intelligence Foundation models with visual question answering capabilities for digital pathology are emerging. Such unprecedented technology requires independent benchmarking to assess its potential in assisting pathologists in routine diagnostics. We created DALPHIN, the first multicentric open benchmark for pathology AI copilots, comprising 1236 images from 300 cases, spanning 130 rare to common diagnoses, 6 countries, and 14 subspecialties. The DALPHIN design and dataset are introduced alongside a human performance benchmark of 31 pathologists from 10 countries with varying expertise. We report results for two general-purpose (GPT-5, Gemini 2.5 Pro) and one pathology-specific copilot (PathChat+) for sequential and independent answer generation. We observed no statistically significant difference from expert-level performance in four of six tasks for PathChat, 2/6 tasks for Gemini, and 1/6 tasks for GPT. DALPHIN is publicly released with sequestered, indirectly accessible ground truth to foster robust and enduring benchmarking. Data, methods, and the evaluation platform are accessible through dalphin.grand-challenge.org. |
| title | DALPHIN: Benchmarking Digital Pathology AI Copilots Against Pathologists on an Open Multicentric Dataset |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2605.03544 |