Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2407.14904 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866918121476980736 |
|---|---|
| author | Shen, Chen Lian, Chunfeng Zhang, Wanqing Wang, Fan Zhang, Jianhua Fan, Shuanliang Wei, Xin Wang, Gongji Li, Kehan Mu, Hongshu Wu, Hao Liang, Xinggong Ma, Jianhua Wang, Zhenyuan |
| author_facet | Shen, Chen Lian, Chunfeng Zhang, Wanqing Wang, Fan Zhang, Jianhua Fan, Shuanliang Wei, Xin Wang, Gongji Li, Kehan Mu, Hongshu Wu, Hao Liang, Xinggong Ma, Jianhua Wang, Zhenyuan |
| contents | Forensic pathology is critical in determining the cause and manner of death through post-mortem examinations, both macroscopic and microscopic. The field, however, grapples with issues such as outcome variability, laborious processes, and a scarcity of trained professionals. This paper presents SongCi, an innovative visual-language model (VLM) designed specifically for forensic pathology. SongCi utilizes advanced prototypical cross-modal self-supervised contrastive learning to enhance the accuracy, efficiency, and generalizability of forensic analyses. It was pre-trained and evaluated on a comprehensive multi-center dataset, which includes over 16 million high-resolution image patches, 2,228 vision-language pairs of post-mortem whole slide images (WSIs), and corresponding gross key findings, along with 471 distinct diagnostic outcomes. Our findings indicate that SongCi surpasses existing multi-modal AI models in many forensic pathology tasks, performs comparably to experienced forensic pathologists and significantly better than less experienced ones, and provides detailed multi-modal explainability, offering critical assistance in forensic investigations. To the best of our knowledge, SongCi is the first VLM specifically developed for forensic pathological analysis and the first large-vocabulary computational pathology (CPath) model that directly processes gigapixel WSIs in forensic science. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_14904 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Large-vocabulary forensic pathological analyses via prototypical cross-modal contrastive learning Shen, Chen Lian, Chunfeng Zhang, Wanqing Wang, Fan Zhang, Jianhua Fan, Shuanliang Wei, Xin Wang, Gongji Li, Kehan Mu, Hongshu Wu, Hao Liang, Xinggong Ma, Jianhua Wang, Zhenyuan Image and Video Processing Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition Forensic pathology is critical in determining the cause and manner of death through post-mortem examinations, both macroscopic and microscopic. The field, however, grapples with issues such as outcome variability, laborious processes, and a scarcity of trained professionals. This paper presents SongCi, an innovative visual-language model (VLM) designed specifically for forensic pathology. SongCi utilizes advanced prototypical cross-modal self-supervised contrastive learning to enhance the accuracy, efficiency, and generalizability of forensic analyses. It was pre-trained and evaluated on a comprehensive multi-center dataset, which includes over 16 million high-resolution image patches, 2,228 vision-language pairs of post-mortem whole slide images (WSIs), and corresponding gross key findings, along with 471 distinct diagnostic outcomes. Our findings indicate that SongCi surpasses existing multi-modal AI models in many forensic pathology tasks, performs comparably to experienced forensic pathologists and significantly better than less experienced ones, and provides detailed multi-modal explainability, offering critical assistance in forensic investigations. To the best of our knowledge, SongCi is the first VLM specifically developed for forensic pathological analysis and the first large-vocabulary computational pathology (CPath) model that directly processes gigapixel WSIs in forensic science. |
| title | Large-vocabulary forensic pathological analyses via prototypical cross-modal contrastive learning |
| topic | Image and Video Processing Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2407.14904 |