The Illusion of Readiness in Health AI
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909957381685248 |
|---|---|
| author | Gu, Yu Fu, Jingjing Liu, Xiaodong Valanarasu, Jeya Maria Jose Codella, Noel CF Tan, Reuben Liu, Qianchu Jin, Ying Zhang, Sheng Wang, Jinyu Wang, Rui Song, Lei Qin, Guanghui Usuyama, Naoto Wong, Cliff Cheng, Hao Lee, HoHin Sanapathi, Praneeth Hilado, Sarah Naumann, Tristan Alvarez-Valle, Javier Bian, Jiang Wei, Mu Malik, Khalil Zhou, Lidong Gao, Jianfeng Horvitz, Eric Lungren, Matthew P. Burger, Doug Topol, Eric Poon, Hoifung Vozila, Paul |
| author_facet | Gu, Yu Fu, Jingjing Liu, Xiaodong Valanarasu, Jeya Maria Jose Codella, Noel CF Tan, Reuben Liu, Qianchu Jin, Ying Zhang, Sheng Wang, Jinyu Wang, Rui Song, Lei Qin, Guanghui Usuyama, Naoto Wong, Cliff Cheng, Hao Lee, HoHin Sanapathi, Praneeth Hilado, Sarah Naumann, Tristan Alvarez-Valle, Javier Bian, Jiang Wei, Mu Malik, Khalil Zhou, Lidong Gao, Jianfeng Horvitz, Eric Lungren, Matthew P. Burger, Doug Topol, Eric Poon, Hoifung Vozila, Paul |
| contents | Large language models have demonstrated remarkable performance in a wide range of medical benchmarks. Yet underneath the seemingly promising results lie salient growth areas, especially in cutting-edge frontiers such as multimodal reasoning. In this paper, we introduce a series of adversarial stress tests to systematically assess the robustness of flagship models and medical benchmarks. Our study reveals prevalent brittleness in the presence of simple adversarial transformations: leading systems can guess the right answer even with key inputs removed, yet may get confused by the slightest prompt alterations, while fabricating convincing yet flawed reasoning traces. Using clinician-guided rubrics, we demonstrate that popular medical benchmarks vary widely in what they truly measure. Our study reveals significant competency gaps of frontier AI in attaining real-world readiness for health applications. If we want AI to earn trust in healthcare, we must demand more than leaderboard wins and must hold AI systems accountable to ensure robustness, sound reasoning, and alignment with real medical demands. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_18234 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | The Illusion of Readiness in Health AI Gu, Yu Fu, Jingjing Liu, Xiaodong Valanarasu, Jeya Maria Jose Codella, Noel CF Tan, Reuben Liu, Qianchu Jin, Ying Zhang, Sheng Wang, Jinyu Wang, Rui Song, Lei Qin, Guanghui Usuyama, Naoto Wong, Cliff Cheng, Hao Lee, HoHin Sanapathi, Praneeth Hilado, Sarah Naumann, Tristan Alvarez-Valle, Javier Bian, Jiang Wei, Mu Malik, Khalil Zhou, Lidong Gao, Jianfeng Horvitz, Eric Lungren, Matthew P. Burger, Doug Topol, Eric Poon, Hoifung Vozila, Paul Artificial Intelligence Computation and Language Machine Learning Large language models have demonstrated remarkable performance in a wide range of medical benchmarks. Yet underneath the seemingly promising results lie salient growth areas, especially in cutting-edge frontiers such as multimodal reasoning. In this paper, we introduce a series of adversarial stress tests to systematically assess the robustness of flagship models and medical benchmarks. Our study reveals prevalent brittleness in the presence of simple adversarial transformations: leading systems can guess the right answer even with key inputs removed, yet may get confused by the slightest prompt alterations, while fabricating convincing yet flawed reasoning traces. Using clinician-guided rubrics, we demonstrate that popular medical benchmarks vary widely in what they truly measure. Our study reveals significant competency gaps of frontier AI in attaining real-world readiness for health applications. If we want AI to earn trust in healthcare, we must demand more than leaderboard wins and must hold AI systems accountable to ensure robustness, sound reasoning, and alignment with real medical demands. |
| title | The Illusion of Readiness in Health AI |
| topic | Artificial Intelligence Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2509.18234 |