Exploring the Feasibility of Multimodal Chatbot AI as Copilot in Pathology Diagnostics: Generalist Model's Pitfall
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912043613814784 |
|---|---|
| author | Liu, Mianxin Wu, Jianfeng Yan, Fang Li, Hongjun Wang, Wei Zhang, Shaoting Wang, Zhe |
| author_facet | Liu, Mianxin Wu, Jianfeng Yan, Fang Li, Hongjun Wang, Wei Zhang, Shaoting Wang, Zhe |
| contents | Pathology images are crucial for diagnosing and managing various diseases by visualizing cellular and tissue-level abnormalities. Recent advancements in artificial intelligence (AI), particularly multimodal models like ChatGPT, have shown promise in transforming medical image analysis through capabilities such as medical vision-language question answering. However, there remains a significant gap in integrating pathology image data with these AI models for clinical applications. This study benchmarks the performance of GPT on pathology images, assessing their diagnostic accuracy and efficiency in real-word clinical records. We observe significant deficits of GPT in bone diseases and a fair-level performance in diseases from other three systems. Despite offering satisfactory abnormality annotations, GPT exhibits consistent disadvantage in terminology accuracy and multimodal integration. Specifically, we demonstrate GPT's failures in interpreting immunohistochemistry results and diagnosing metastatic cancers. This study highlight the weakness of current generalist GPT model and contribute to the integration of pathology and advanced AI. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_15291 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Exploring the Feasibility of Multimodal Chatbot AI as Copilot in Pathology Diagnostics: Generalist Model's Pitfall Liu, Mianxin Wu, Jianfeng Yan, Fang Li, Hongjun Wang, Wei Zhang, Shaoting Wang, Zhe Human-Computer Interaction Computers and Society Pathology images are crucial for diagnosing and managing various diseases by visualizing cellular and tissue-level abnormalities. Recent advancements in artificial intelligence (AI), particularly multimodal models like ChatGPT, have shown promise in transforming medical image analysis through capabilities such as medical vision-language question answering. However, there remains a significant gap in integrating pathology image data with these AI models for clinical applications. This study benchmarks the performance of GPT on pathology images, assessing their diagnostic accuracy and efficiency in real-word clinical records. We observe significant deficits of GPT in bone diseases and a fair-level performance in diseases from other three systems. Despite offering satisfactory abnormality annotations, GPT exhibits consistent disadvantage in terminology accuracy and multimodal integration. Specifically, we demonstrate GPT's failures in interpreting immunohistochemistry results and diagnosing metastatic cancers. This study highlight the weakness of current generalist GPT model and contribute to the integration of pathology and advanced AI. |
| title | Exploring the Feasibility of Multimodal Chatbot AI as Copilot in Pathology Diagnostics: Generalist Model's Pitfall |
| topic | Human-Computer Interaction Computers and Society |
| url | https://arxiv.org/abs/2409.15291 |