Establishing Rigorous and Cost-effective Clinical Trials for Artificial Intelligence Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916338541264896 |
|---|---|
| author | Gao, Wanling Huang, Yunyou Cui, Dandan Yu, Zhuoming Liu, Wenjing Liang, Xiaoshuang Zhao, Jiahui Xie, Jiyue Li, Hao Ma, Li Ye, Ning Kang, Yumiao Luo, Dingfeng Pan, Peng Huang, Wei Liu, Zhongmou Hu, Jizhong Zhao, Gangyuan Jiang, Chongrong Huang, Fan Wei, Tianyi Tang, Suqin Xia, Bingjie Zhang, Zhifei Zhan, Jianfeng |
| author_facet | Gao, Wanling Huang, Yunyou Cui, Dandan Yu, Zhuoming Liu, Wenjing Liang, Xiaoshuang Zhao, Jiahui Xie, Jiyue Li, Hao Ma, Li Ye, Ning Kang, Yumiao Luo, Dingfeng Pan, Peng Huang, Wei Liu, Zhongmou Hu, Jizhong Zhao, Gangyuan Jiang, Chongrong Huang, Fan Wei, Tianyi Tang, Suqin Xia, Bingjie Zhang, Zhifei Zhan, Jianfeng |
| contents | A profound gap persists between artificial intelligence (AI) and clinical practice in medicine, primarily due to the lack of rigorous and cost-effective evaluation methodologies. State-of-the-art and state-of-the-practice AI model evaluations are limited to laboratory studies on medical datasets or direct clinical trials with no or solely patient-centered controls. Moreover, the crucial role of clinicians in collaborating with AI, pivotal for determining its impact on clinical practice, is often overlooked. For the first time, we emphasize the critical necessity for rigorous and cost-effective evaluation methodologies for AI models in clinical practice, featuring patient/clinician-centered (dual-centered) AI randomized controlled trials (DC-AI RCTs) and virtual clinician-based in-silico trials (VC-MedAI) as an effective proxy for DC-AI RCTs. Leveraging 7500 diagnosis records from two-step inaugural DC-AI RCTs across 14 medical centers with 125 clinicians, our results demonstrate the necessity of DC-AI RCTs and the effectiveness of VC-MedAI. Notably, VC-MedAI performs comparably to human clinicians, replicating insights and conclusions from prospective DC-AI RCTs. We envision DC-AI RCTs and VC-MedAI as pivotal advancements, presenting innovative and transformative evaluation methodologies for AI models in clinical practice, offering a preclinical-like setting mirroring conventional medicine, and reshaping development paradigms in a cost-effective and fast-iterative manner. Chinese Clinical Trial Registration: ChiCTR2400086816. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_08554 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Establishing Rigorous and Cost-effective Clinical Trials for Artificial Intelligence Models Gao, Wanling Huang, Yunyou Cui, Dandan Yu, Zhuoming Liu, Wenjing Liang, Xiaoshuang Zhao, Jiahui Xie, Jiyue Li, Hao Ma, Li Ye, Ning Kang, Yumiao Luo, Dingfeng Pan, Peng Huang, Wei Liu, Zhongmou Hu, Jizhong Zhao, Gangyuan Jiang, Chongrong Huang, Fan Wei, Tianyi Tang, Suqin Xia, Bingjie Zhang, Zhifei Zhan, Jianfeng Artificial Intelligence Human-Computer Interaction A profound gap persists between artificial intelligence (AI) and clinical practice in medicine, primarily due to the lack of rigorous and cost-effective evaluation methodologies. State-of-the-art and state-of-the-practice AI model evaluations are limited to laboratory studies on medical datasets or direct clinical trials with no or solely patient-centered controls. Moreover, the crucial role of clinicians in collaborating with AI, pivotal for determining its impact on clinical practice, is often overlooked. For the first time, we emphasize the critical necessity for rigorous and cost-effective evaluation methodologies for AI models in clinical practice, featuring patient/clinician-centered (dual-centered) AI randomized controlled trials (DC-AI RCTs) and virtual clinician-based in-silico trials (VC-MedAI) as an effective proxy for DC-AI RCTs. Leveraging 7500 diagnosis records from two-step inaugural DC-AI RCTs across 14 medical centers with 125 clinicians, our results demonstrate the necessity of DC-AI RCTs and the effectiveness of VC-MedAI. Notably, VC-MedAI performs comparably to human clinicians, replicating insights and conclusions from prospective DC-AI RCTs. We envision DC-AI RCTs and VC-MedAI as pivotal advancements, presenting innovative and transformative evaluation methodologies for AI models in clinical practice, offering a preclinical-like setting mirroring conventional medicine, and reshaping development paradigms in a cost-effective and fast-iterative manner. Chinese Clinical Trial Registration: ChiCTR2400086816. |
| title | Establishing Rigorous and Cost-effective Clinical Trials for Artificial Intelligence Models |
| topic | Artificial Intelligence Human-Computer Interaction |
| url | https://arxiv.org/abs/2407.08554 |