Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2601.12974 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917210107150336 |
|---|---|
| author | Ma, Hongyang Gu, Tiantian Sun, Huaiyuan Zhu, Huilin Wang, Yongxin Li, Jie Sun, Wubin Lian, Zeliang Zhou, Yinghong Gao, Yi Wang, Shirui Tang, Zhihui |
| author_facet | Ma, Hongyang Gu, Tiantian Sun, Huaiyuan Zhu, Huilin Wang, Yongxin Li, Jie Sun, Wubin Lian, Zeliang Zhou, Yinghong Gao, Yi Wang, Shirui Tang, Zhihui |
| contents | The transition of Large Language Models (LLMs) from passive knowledge retrievers to autonomous clinical agents demands a shift in evaluation-from static accuracy to dynamic behavioral reliability. To explore this boundary in dentistry, a domain where high-quality AI advice uniquely empowers patient-participatory decision-making, we present the Standardized Clinical Management & Performance Evaluation (SCMPE) benchmark, which comprehensively assesses performance from knowledge-oriented evaluations (static objective tasks) to workflow-based simulations (multi-turn simulated patient interactions). Our analysis reveals that while models demonstrate high proficiency in static objective tasks, their performance precipitates in dynamic clinical dialogues, identifying that the primary bottleneck lies not in knowledge retention, but in the critical challenges of active information gathering and dynamic state tracking. Mapping "Guideline Adherence" versus "Decision Quality" reveals a prevalent "High Efficacy, Low Safety" risk in general models. Furthermore, we quantify the impact of Retrieval-Augmented Generation (RAG). While RAG mitigates hallucinations in static tasks, its efficacy in dynamic workflows is limited and heterogeneous, sometimes causing degradation. This underscores that external knowledge alone cannot bridge the reasoning gap without domain-adaptive pre-training. This study empirically charts the capability boundaries of dental LLMs, providing a roadmap for bridging the gap between standardized knowledge and safe, autonomous clinical practice. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_12974 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Bridging the Knowledge-Action Gap by Evaluating LLMs in Dynamic Dental Clinical Scenarios Ma, Hongyang Gu, Tiantian Sun, Huaiyuan Zhu, Huilin Wang, Yongxin Li, Jie Sun, Wubin Lian, Zeliang Zhou, Yinghong Gao, Yi Wang, Shirui Tang, Zhihui Computation and Language The transition of Large Language Models (LLMs) from passive knowledge retrievers to autonomous clinical agents demands a shift in evaluation-from static accuracy to dynamic behavioral reliability. To explore this boundary in dentistry, a domain where high-quality AI advice uniquely empowers patient-participatory decision-making, we present the Standardized Clinical Management & Performance Evaluation (SCMPE) benchmark, which comprehensively assesses performance from knowledge-oriented evaluations (static objective tasks) to workflow-based simulations (multi-turn simulated patient interactions). Our analysis reveals that while models demonstrate high proficiency in static objective tasks, their performance precipitates in dynamic clinical dialogues, identifying that the primary bottleneck lies not in knowledge retention, but in the critical challenges of active information gathering and dynamic state tracking. Mapping "Guideline Adherence" versus "Decision Quality" reveals a prevalent "High Efficacy, Low Safety" risk in general models. Furthermore, we quantify the impact of Retrieval-Augmented Generation (RAG). While RAG mitigates hallucinations in static tasks, its efficacy in dynamic workflows is limited and heterogeneous, sometimes causing degradation. This underscores that external knowledge alone cannot bridge the reasoning gap without domain-adaptive pre-training. This study empirically charts the capability boundaries of dental LLMs, providing a roadmap for bridging the gap between standardized knowledge and safe, autonomous clinical practice. |
| title | Bridging the Knowledge-Action Gap by Evaluating LLMs in Dynamic Dental Clinical Scenarios |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2601.12974 |