Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2505.08414 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916735115853824 |
|---|---|
| author | Da Soh, Zhi Bai, Yang Yu, Kai Zhou, Yang Lei, Xiaofeng Thakur, Sahil Lee, Zann Phang, Lee Ching Linette Peng, Qingsheng Xue, Can Can Chong, Rachel Shujuan Hoang, Quan V. Raghavan, Lavanya Tham, Yih Chung Sabanayagam, Charumathi Wu, Wei-Chi Ho, Ming-Chih He, Jiangnan Gupta, Preeti Lamoureux, Ecosse Saw, Seang Mei Nangia, Vinay Panda-Jonas, Songhomitra Xu, Jie Wang, Ya Xing Xu, Xinxing Jonas, Jost B. Wong, Tien Yin Goh, Rick Siow Mong Liu, Yong Cheng, Ching-Yu |
| author_facet | Da Soh, Zhi Bai, Yang Yu, Kai Zhou, Yang Lei, Xiaofeng Thakur, Sahil Lee, Zann Phang, Lee Ching Linette Peng, Qingsheng Xue, Can Can Chong, Rachel Shujuan Hoang, Quan V. Raghavan, Lavanya Tham, Yih Chung Sabanayagam, Charumathi Wu, Wei-Chi Ho, Ming-Chih He, Jiangnan Gupta, Preeti Lamoureux, Ecosse Saw, Seang Mei Nangia, Vinay Panda-Jonas, Songhomitra Xu, Jie Wang, Ya Xing Xu, Xinxing Jonas, Jost B. Wong, Tien Yin Goh, Rick Siow Mong Liu, Yong Cheng, Ching-Yu |
| contents | Current deep learning models are mostly task specific and lack a user-friendly interface to operate. We present Meta-EyeFM, a multi-function foundation model that integrates a large language model (LLM) with vision foundation models (VFMs) for ocular disease assessment. Meta-EyeFM leverages a routing mechanism to enable accurate task-specific analysis based on text queries. Using Low Rank Adaptation, we fine-tuned our VFMs to detect ocular and systemic diseases, differentiate ocular disease severity, and identify common ocular signs. The model achieved 100% accuracy in routing fundus images to appropriate VFMs, which achieved $\ge$ 82.2% accuracy in disease detection, $\ge$ 89% in severity differentiation, $\ge$ 76% in sign identification. Meta-EyeFM was 11% to 43% more accurate than Gemini-1.5-flash and ChatGPT-4o LMMs in detecting various eye diseases and comparable to an ophthalmologist. This system offers enhanced usability and diagnostic performance, making it a valuable decision support tool for primary eye care or an online LLM for fundus evaluation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_08414 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | An integrated language-vision foundation model for conversational diagnostics and triaging in primary eye care Da Soh, Zhi Bai, Yang Yu, Kai Zhou, Yang Lei, Xiaofeng Thakur, Sahil Lee, Zann Phang, Lee Ching Linette Peng, Qingsheng Xue, Can Can Chong, Rachel Shujuan Hoang, Quan V. Raghavan, Lavanya Tham, Yih Chung Sabanayagam, Charumathi Wu, Wei-Chi Ho, Ming-Chih He, Jiangnan Gupta, Preeti Lamoureux, Ecosse Saw, Seang Mei Nangia, Vinay Panda-Jonas, Songhomitra Xu, Jie Wang, Ya Xing Xu, Xinxing Jonas, Jost B. Wong, Tien Yin Goh, Rick Siow Mong Liu, Yong Cheng, Ching-Yu Image and Video Processing Computer Vision and Pattern Recognition Current deep learning models are mostly task specific and lack a user-friendly interface to operate. We present Meta-EyeFM, a multi-function foundation model that integrates a large language model (LLM) with vision foundation models (VFMs) for ocular disease assessment. Meta-EyeFM leverages a routing mechanism to enable accurate task-specific analysis based on text queries. Using Low Rank Adaptation, we fine-tuned our VFMs to detect ocular and systemic diseases, differentiate ocular disease severity, and identify common ocular signs. The model achieved 100% accuracy in routing fundus images to appropriate VFMs, which achieved $\ge$ 82.2% accuracy in disease detection, $\ge$ 89% in severity differentiation, $\ge$ 76% in sign identification. Meta-EyeFM was 11% to 43% more accurate than Gemini-1.5-flash and ChatGPT-4o LMMs in detecting various eye diseases and comparable to an ophthalmologist. This system offers enhanced usability and diagnostic performance, making it a valuable decision support tool for primary eye care or an online LLM for fundus evaluation. |
| title | An integrated language-vision foundation model for conversational diagnostics and triaging in primary eye care |
| topic | Image and Video Processing Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2505.08414 |