Towards Democratization of Subspeciality Medical Expertise
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912060008300544 |
|---|---|
| author | O'Sullivan, Jack W. Palepu, Anil Saab, Khaled Weng, Wei-Hung Cheng, Yong Chu, Emily Desai, Yaanik Elezaby, Aly Kim, Daniel Seung Lan, Roy Tang, Wilson Tapaskar, Natalie Parikh, Victoria Jain, Sneha S. Kulkarni, Kavita Mansfield, Philip Webster, Dale Gottweis, Juraj Barral, Joelle Schaekermann, Mike Tanno, Ryutaro Mahdavi, S. Sara Natarajan, Vivek Karthikesalingam, Alan Ashley, Euan Tu, Tao |
| author_facet | O'Sullivan, Jack W. Palepu, Anil Saab, Khaled Weng, Wei-Hung Cheng, Yong Chu, Emily Desai, Yaanik Elezaby, Aly Kim, Daniel Seung Lan, Roy Tang, Wilson Tapaskar, Natalie Parikh, Victoria Jain, Sneha S. Kulkarni, Kavita Mansfield, Philip Webster, Dale Gottweis, Juraj Barral, Joelle Schaekermann, Mike Tanno, Ryutaro Mahdavi, S. Sara Natarajan, Vivek Karthikesalingam, Alan Ashley, Euan Tu, Tao |
| contents | The scarcity of subspecialist medical expertise, particularly in rare, complex and life-threatening diseases, poses a significant challenge for healthcare delivery. This issue is particularly acute in cardiology where timely, accurate management determines outcomes. We explored the potential of AMIE (Articulate Medical Intelligence Explorer), a large language model (LLM)-based experimental AI system optimized for diagnostic dialogue, to potentially augment and support clinical decision-making in this challenging context. We curated a real-world dataset of 204 complex cases from a subspecialist cardiology practice, including results for electrocardiograms, echocardiograms, cardiac MRI, genetic tests, and cardiopulmonary stress tests. We developed a ten-domain evaluation rubric used by subspecialists to evaluate the quality of diagnosis and clinical management plans produced by general cardiologists or AMIE, the latter enhanced with web-search and self-critique capabilities. AMIE was rated superior to general cardiologists for 5 of the 10 domains (with preference ranging from 9% to 20%), and equivalent for the rest. Access to AMIE's response improved cardiologists' overall response quality in 63.7% of cases while lowering quality in just 3.4%. Cardiologists' responses with access to AMIE were superior to cardiologist responses without access to AMIE for all 10 domains. Qualitative examinations suggest AMIE and general cardiologist could complement each other, with AMIE thorough and sensitive, while general cardiologist concise and specific. Overall, our results suggest that specialized medical LLMs have the potential to augment general cardiologists' capabilities by bridging gaps in subspecialty expertise, though further research and validation are essential for wide clinical utility. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_03741 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Towards Democratization of Subspeciality Medical Expertise O'Sullivan, Jack W. Palepu, Anil Saab, Khaled Weng, Wei-Hung Cheng, Yong Chu, Emily Desai, Yaanik Elezaby, Aly Kim, Daniel Seung Lan, Roy Tang, Wilson Tapaskar, Natalie Parikh, Victoria Jain, Sneha S. Kulkarni, Kavita Mansfield, Philip Webster, Dale Gottweis, Juraj Barral, Joelle Schaekermann, Mike Tanno, Ryutaro Mahdavi, S. Sara Natarajan, Vivek Karthikesalingam, Alan Ashley, Euan Tu, Tao Human-Computer Interaction Artificial Intelligence The scarcity of subspecialist medical expertise, particularly in rare, complex and life-threatening diseases, poses a significant challenge for healthcare delivery. This issue is particularly acute in cardiology where timely, accurate management determines outcomes. We explored the potential of AMIE (Articulate Medical Intelligence Explorer), a large language model (LLM)-based experimental AI system optimized for diagnostic dialogue, to potentially augment and support clinical decision-making in this challenging context. We curated a real-world dataset of 204 complex cases from a subspecialist cardiology practice, including results for electrocardiograms, echocardiograms, cardiac MRI, genetic tests, and cardiopulmonary stress tests. We developed a ten-domain evaluation rubric used by subspecialists to evaluate the quality of diagnosis and clinical management plans produced by general cardiologists or AMIE, the latter enhanced with web-search and self-critique capabilities. AMIE was rated superior to general cardiologists for 5 of the 10 domains (with preference ranging from 9% to 20%), and equivalent for the rest. Access to AMIE's response improved cardiologists' overall response quality in 63.7% of cases while lowering quality in just 3.4%. Cardiologists' responses with access to AMIE were superior to cardiologist responses without access to AMIE for all 10 domains. Qualitative examinations suggest AMIE and general cardiologist could complement each other, with AMIE thorough and sensitive, while general cardiologist concise and specific. Overall, our results suggest that specialized medical LLMs have the potential to augment general cardiologists' capabilities by bridging gaps in subspecialty expertise, though further research and validation are essential for wide clinical utility. |
| title | Towards Democratization of Subspeciality Medical Expertise |
| topic | Human-Computer Interaction Artificial Intelligence |
| url | https://arxiv.org/abs/2410.03741 |