Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Artículo Open Access |
| Veröffentlicht: |
Wiley
2024
|
| Schlagworte: | |
| Online-Zugang: | https://onlinelibrary.wiley.com/doi/10.1002/imo2.7 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1867014560674742272 |
|---|---|
| author | Jinghong Chen Lingxuan Zhu Weiming Mou Anqi Lin Dongqiang Zeng Chang Qi Zaoqu Liu Aimin Jiang Bufu Tang Wenjie Shi Ulf D. Kahlert Jianguo Zhou Shipeng Guo Xiaofan Lu Xu Sun Trunghieu Ngo Zhongji Pu Baolei Jia Che Ok Jeon Yongbin He Haiyang Wu Shuqin Gu Wisit Cheungpasitporn Haojie Huang Weipu Mao Shixiang Wang Xin Chen Loïc Cabannes Gerald Sng Gui Ren Iain S. Whitaker Stephen Ali Quan Cheng Kai Miao Shuofeng Yuan Peng Luo |
| author_facet | Jinghong Chen Lingxuan Zhu Weiming Mou Anqi Lin Dongqiang Zeng Chang Qi Zaoqu Liu Aimin Jiang Bufu Tang Wenjie Shi Ulf D. Kahlert Jianguo Zhou Shipeng Guo Xiaofan Lu Xu Sun Trunghieu Ngo Zhongji Pu Baolei Jia Che Ok Jeon Yongbin He Haiyang Wu Shuqin Gu Wisit Cheungpasitporn Haojie Huang Weipu Mao Shixiang Wang Xin Chen Loïc Cabannes Gerald Sng Gui Ren Iain S. Whitaker Stephen Ali Quan Cheng Kai Miao Shuofeng Yuan Peng Luo Jinghong Chen Lingxuan Zhu Weiming Mou Anqi Lin Dongqiang Zeng Chang Qi Zaoqu Liu Aimin Jiang Bufu Tang Wenjie Shi Ulf D. Kahlert Jianguo Zhou Shipeng Guo Xiaofan Lu Xu Sun Trunghieu Ngo Zhongji Pu Baolei Jia Che Ok Jeon Yongbin He Haiyang Wu Shuqin Gu Wisit Cheungpasitporn Haojie Huang Weipu Mao Shixiang Wang Xin Chen Loïc Cabannes Gerald Sng Gui Ren Iain S. Whitaker Stephen Ali Quan Cheng Kai Miao Shuofeng Yuan Peng Luo |
| collection | Wiley Open Access |
| contents | STAGER checklist: Standardized testing and assessment guidelines for evaluating generative artificial intelligence reliability Jinghong Chen Lingxuan Zhu Weiming Mou Anqi Lin Dongqiang Zeng Chang Qi Zaoqu Liu Aimin Jiang Bufu Tang Wenjie Shi Ulf D. Kahlert Jianguo Zhou Shipeng Guo Xiaofan Lu Xu Sun Trunghieu Ngo Zhongji Pu Baolei Jia Che Ok Jeon Yongbin He Haiyang Wu Shuqin Gu Wisit Cheungpasitporn Haojie Huang Weipu Mao Shixiang Wang Xin Chen Loïc Cabannes Gerald Sng Gui Ren Iain S. Whitaker Stephen Ali Quan Cheng Kai Miao Shuofeng Yuan Peng Luo iMetaOmics AbstractGenerative artificial intelligence (AI) holds immense potential for medical applications, but the lack of a comprehensive evaluation framework and methodological deficiencies in existing studies hinder its effective implementation. Standardized assessment guidelines are crucial for ensuring reliable and consistent evaluation of generative AI in healthcare. Our objective is to develop robust, standardized guidelines tailored for evaluating generative AI performance in medical contexts. Through a rigorous literature review utilizing the Web of Sciences, Cochrane Library, PubMed, and Google Scholar, we focused on research testing generative AI capabilities in medicine. Our multidisciplinary team of experts conducted discussion sessions to develop a comprehensive 32‐item checklist. This checklist encompasses critical evaluation aspects of generative AI in medical applications, addressing key dimensions such as question collection, querying methodologies, and assessment techniques. The checklist and its broader assessment framework provide a holistic evaluation of AI systems, delineating a clear pathway from question gathering to result assessment. It guides researchers through potential challenges and pitfalls, enhancing research quality and reporting and aiding the evolution of generative AI in medicine and life sciences. Our framework furnishes a standardized, systematic approach for testing generative AI's applicability in medicine. For a concise checklist, please refer to Table S or visit GenAIMed.org. 10.1002/imo2.7 http://creativecommons.org/licenses/by/4.0/ |
| doi_str_mv | 10.1002/imo2.7 |
| format | Artículo Open Access |
| id | wiley_oa_10_1002_imo2_7 |
| institution | Wiley Open Access |
| license_str_mv | http://creativecommons.org/licenses/by/4.0/ |
| publishDate | 2024 |
| publisher | Wiley |
| record_format | wiley_oa |
| spellingShingle | STAGER checklist: Standardized testing and assessment guidelines for evaluating generative artificial intelligence reliability Jinghong Chen Lingxuan Zhu Weiming Mou Anqi Lin Dongqiang Zeng Chang Qi Zaoqu Liu Aimin Jiang Bufu Tang Wenjie Shi Ulf D. Kahlert Jianguo Zhou Shipeng Guo Xiaofan Lu Xu Sun Trunghieu Ngo Zhongji Pu Baolei Jia Che Ok Jeon Yongbin He Haiyang Wu Shuqin Gu Wisit Cheungpasitporn Haojie Huang Weipu Mao Shixiang Wang Xin Chen Loïc Cabannes Gerald Sng Gui Ren Iain S. Whitaker Stephen Ali Quan Cheng Kai Miao Shuofeng Yuan Peng Luo iMetaOmics STAGER checklist: Standardized testing and assessment guidelines for evaluating generative artificial intelligence reliability Jinghong Chen Lingxuan Zhu Weiming Mou Anqi Lin Dongqiang Zeng Chang Qi Zaoqu Liu Aimin Jiang Bufu Tang Wenjie Shi Ulf D. Kahlert Jianguo Zhou Shipeng Guo Xiaofan Lu Xu Sun Trunghieu Ngo Zhongji Pu Baolei Jia Che Ok Jeon Yongbin He Haiyang Wu Shuqin Gu Wisit Cheungpasitporn Haojie Huang Weipu Mao Shixiang Wang Xin Chen Loïc Cabannes Gerald Sng Gui Ren Iain S. Whitaker Stephen Ali Quan Cheng Kai Miao Shuofeng Yuan Peng Luo iMetaOmics AbstractGenerative artificial intelligence (AI) holds immense potential for medical applications, but the lack of a comprehensive evaluation framework and methodological deficiencies in existing studies hinder its effective implementation. Standardized assessment guidelines are crucial for ensuring reliable and consistent evaluation of generative AI in healthcare. Our objective is to develop robust, standardized guidelines tailored for evaluating generative AI performance in medical contexts. Through a rigorous literature review utilizing the Web of Sciences, Cochrane Library, PubMed, and Google Scholar, we focused on research testing generative AI capabilities in medicine. Our multidisciplinary team of experts conducted discussion sessions to develop a comprehensive 32‐item checklist. This checklist encompasses critical evaluation aspects of generative AI in medical applications, addressing key dimensions such as question collection, querying methodologies, and assessment techniques. The checklist and its broader assessment framework provide a holistic evaluation of AI systems, delineating a clear pathway from question gathering to result assessment. It guides researchers through potential challenges and pitfalls, enhancing research quality and reporting and aiding the evolution of generative AI in medicine and life sciences. Our framework furnishes a standardized, systematic approach for testing generative AI's applicability in medicine. For a concise checklist, please refer to Table S or visit GenAIMed.org. 10.1002/imo2.7 http://creativecommons.org/licenses/by/4.0/ |
| title | STAGER checklist: Standardized testing and assessment guidelines for evaluating generative artificial intelligence reliability |
| topic | iMetaOmics |
| url | https://onlinelibrary.wiley.com/doi/10.1002/imo2.7 |