Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2406.09317 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913799565475840 |
|---|---|
| author | Wang, Meng Lin, Tian Lin, Aidi Yu, Kai Peng, Yuanyuan Wang, Lianyu Chen, Cheng Zou, Ke Liang, Huiyu Chen, Man Yao, Xue Zhang, Meiqin Huang, Binwei Zheng, Chaoxin Zhang, Peixin Chen, Wei Luo, Yilong Chen, Yifan Xia, Honghe Shi, Tingkun Zhang, Qi Guo, Jinming Chen, Xiaolin Wang, Jingcheng Tham, Yih Chung Liu, Dianbo Wong, Wendy Thakur, Sahil Fenner, Beau Fang, Danqi Liu, Siying Liu, Qingyun Huang, Yuqiang Zeng, Hongqiang Meng, Yanda Zhou, Yukun Jiang, Zehua Qiu, Minghui Zhang, Changqing Chen, Xinjian Wang, Sophia Y. Lee, Cecilia S. Sobrin, Lucia Cheung, Carol Y Pang, Chi Pui Keane, Pearse A. Cheng, Ching-Yu Chen, Haoyu Fu, Huazhu |
| author_facet | Wang, Meng Lin, Tian Lin, Aidi Yu, Kai Peng, Yuanyuan Wang, Lianyu Chen, Cheng Zou, Ke Liang, Huiyu Chen, Man Yao, Xue Zhang, Meiqin Huang, Binwei Zheng, Chaoxin Zhang, Peixin Chen, Wei Luo, Yilong Chen, Yifan Xia, Honghe Shi, Tingkun Zhang, Qi Guo, Jinming Chen, Xiaolin Wang, Jingcheng Tham, Yih Chung Liu, Dianbo Wong, Wendy Thakur, Sahil Fenner, Beau Fang, Danqi Liu, Siying Liu, Qingyun Huang, Yuqiang Zeng, Hongqiang Meng, Yanda Zhou, Yukun Jiang, Zehua Qiu, Minghui Zhang, Changqing Chen, Xinjian Wang, Sophia Y. Lee, Cecilia S. Sobrin, Lucia Cheung, Carol Y Pang, Chi Pui Keane, Pearse A. Cheng, Ching-Yu Chen, Haoyu Fu, Huazhu |
| contents | Previous foundation models for fundus images were pre-trained with limited disease categories and knowledge base. Here we introduce a knowledge-rich vision-language model (RetiZero) that leverages knowledge from more than 400 fundus diseases. For RetiZero's pretraining, we compiled 341,896 fundus images paired with texts, sourced from public datasets, ophthalmic literature, and online resources, encompassing a diverse range of diseases across multiple ethnicities and countries. RetiZero exhibits remarkable performance in several downstream tasks, including zero-shot disease recognition, image-to-image retrieval, AI-assisted clinical diagnosis,few-shot fine-tuning, and internal- and cross-domain disease identification. In zero-shot scenarios, RetiZero achieves Top-5 accuracies of 0.843 for 15 diseases and 0.756 for 52 diseases. For image retrieval, it achieves Top-5 scores of 0.950 and 0.886 for the same sets, respectively. AI-assisted clinical diagnosis results show that RetiZero's Top-3 zero-shot performance surpasses the average of 19 ophthalmologists from Singapore, China, and the United States. RetiZero substantially enhances clinicians' accuracy in diagnosing fundus diseases, in particularly rare ones. These findings underscore the value of integrating the RetiZero into clinical settings, where various fundus diseases are encountered. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_09317 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Enhancing Diagnostic Accuracy in Rare and Common Fundus Diseases with a Knowledge-Rich Vision-Language Model Wang, Meng Lin, Tian Lin, Aidi Yu, Kai Peng, Yuanyuan Wang, Lianyu Chen, Cheng Zou, Ke Liang, Huiyu Chen, Man Yao, Xue Zhang, Meiqin Huang, Binwei Zheng, Chaoxin Zhang, Peixin Chen, Wei Luo, Yilong Chen, Yifan Xia, Honghe Shi, Tingkun Zhang, Qi Guo, Jinming Chen, Xiaolin Wang, Jingcheng Tham, Yih Chung Liu, Dianbo Wong, Wendy Thakur, Sahil Fenner, Beau Fang, Danqi Liu, Siying Liu, Qingyun Huang, Yuqiang Zeng, Hongqiang Meng, Yanda Zhou, Yukun Jiang, Zehua Qiu, Minghui Zhang, Changqing Chen, Xinjian Wang, Sophia Y. Lee, Cecilia S. Sobrin, Lucia Cheung, Carol Y Pang, Chi Pui Keane, Pearse A. Cheng, Ching-Yu Chen, Haoyu Fu, Huazhu Image and Video Processing Computer Vision and Pattern Recognition Previous foundation models for fundus images were pre-trained with limited disease categories and knowledge base. Here we introduce a knowledge-rich vision-language model (RetiZero) that leverages knowledge from more than 400 fundus diseases. For RetiZero's pretraining, we compiled 341,896 fundus images paired with texts, sourced from public datasets, ophthalmic literature, and online resources, encompassing a diverse range of diseases across multiple ethnicities and countries. RetiZero exhibits remarkable performance in several downstream tasks, including zero-shot disease recognition, image-to-image retrieval, AI-assisted clinical diagnosis,few-shot fine-tuning, and internal- and cross-domain disease identification. In zero-shot scenarios, RetiZero achieves Top-5 accuracies of 0.843 for 15 diseases and 0.756 for 52 diseases. For image retrieval, it achieves Top-5 scores of 0.950 and 0.886 for the same sets, respectively. AI-assisted clinical diagnosis results show that RetiZero's Top-3 zero-shot performance surpasses the average of 19 ophthalmologists from Singapore, China, and the United States. RetiZero substantially enhances clinicians' accuracy in diagnosing fundus diseases, in particularly rare ones. These findings underscore the value of integrating the RetiZero into clinical settings, where various fundus diseases are encountered. |
| title | Enhancing Diagnostic Accuracy in Rare and Common Fundus Diseases with a Knowledge-Rich Vision-Language Model |
| topic | Image and Video Processing Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2406.09317 |