Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.17764 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914104470405120 |
|---|---|
| author | Ye, Xiao Dineen, Jacob Li, Zhaonan Xu, Zhikun Chen, Weiyu Lu, Shijie Huang, Yuxi Shen, Ming Tran, Phu Yum, Ji-Eun Irene Khan, Muhammad Ali Afzal, Muhammad Umar Riaz, Irbaz Bin Zhou, Ben |
| author_facet | Ye, Xiao Dineen, Jacob Li, Zhaonan Xu, Zhikun Chen, Weiyu Lu, Shijie Huang, Yuxi Shen, Ming Tran, Phu Yum, Ji-Eun Irene Khan, Muhammad Ali Afzal, Muhammad Umar Riaz, Irbaz Bin Zhou, Ben |
| contents | Medical Large language models achieve strong scores on standard benchmarks; however, the transfer of those results to safe and reliable performance in clinical workflows remains a challenge. This survey reframes evaluation through a levels-of-autonomy lens (L0-L3), spanning informational tools, information transformation and aggregation, decision support, and supervised agents. We align existing benchmarks and metrics with the actions permitted at each level and their associated risks, making the evaluation targets explicit. This motivates a level-conditioned blueprint for selecting metrics, assembling evidence, and reporting claims, alongside directions that link evaluation to oversight. By centering autonomy, the survey moves the field beyond score-based claims toward credible, risk-aware evidence for real clinical use. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_17764 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Evaluating Medical LLMs by Levels of Autonomy: A Survey Moving from Benchmarks to Applications Ye, Xiao Dineen, Jacob Li, Zhaonan Xu, Zhikun Chen, Weiyu Lu, Shijie Huang, Yuxi Shen, Ming Tran, Phu Yum, Ji-Eun Irene Khan, Muhammad Ali Afzal, Muhammad Umar Riaz, Irbaz Bin Zhou, Ben Computation and Language Medical Large language models achieve strong scores on standard benchmarks; however, the transfer of those results to safe and reliable performance in clinical workflows remains a challenge. This survey reframes evaluation through a levels-of-autonomy lens (L0-L3), spanning informational tools, information transformation and aggregation, decision support, and supervised agents. We align existing benchmarks and metrics with the actions permitted at each level and their associated risks, making the evaluation targets explicit. This motivates a level-conditioned blueprint for selecting metrics, assembling evidence, and reporting claims, alongside directions that link evaluation to oversight. By centering autonomy, the survey moves the field beyond score-based claims toward credible, risk-aware evidence for real clinical use. |
| title | Evaluating Medical LLMs by Levels of Autonomy: A Survey Moving from Benchmarks to Applications |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2510.17764 |