Dialogue is Better Than Monologue: Instructing Medical LLMs via Strategical Conversations
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908844793266176 |
|---|---|
| author | Liu, Zijie Zhao, Xinyu Peng, Jie Zhu, Zhuangdi Chen, Qingyu Xu, Kaidi Hu, Xia Chen, Tianlong |
| author_facet | Liu, Zijie Zhao, Xinyu Peng, Jie Zhu, Zhuangdi Chen, Qingyu Xu, Kaidi Hu, Xia Chen, Tianlong |
| contents | Current medical AI systems often fail to replicate real-world clinical reasoning, as they are predominantly trained and evaluated on static text and question-answer tasks. These tuning methods and benchmarks overlook critical aspects like evidence-based reasoning and handling distracting information. To bridge this gap, we introduce a novel benchmark that simulates real-world diagnostic scenarios, integrating noise and difficulty levels aligned with USMLE standards. Moreover, we explore dialogue-based fine-tuning, which transforms static datasets into conversational formats to better capture iterative reasoning processes. Experiments show that dialogue-tuned models outperform traditional methods, with improvements of $9.64\%$ in multi-round reasoning scenarios and $6.18\%$ in accuracy in a noisy environment. Our findings highlight dialogue tuning as a promising approach for advancing clinically aligned and robust medical AI systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_17860 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Dialogue is Better Than Monologue: Instructing Medical LLMs via Strategical Conversations Liu, Zijie Zhao, Xinyu Peng, Jie Zhu, Zhuangdi Chen, Qingyu Xu, Kaidi Hu, Xia Chen, Tianlong Computation and Language Artificial Intelligence Current medical AI systems often fail to replicate real-world clinical reasoning, as they are predominantly trained and evaluated on static text and question-answer tasks. These tuning methods and benchmarks overlook critical aspects like evidence-based reasoning and handling distracting information. To bridge this gap, we introduce a novel benchmark that simulates real-world diagnostic scenarios, integrating noise and difficulty levels aligned with USMLE standards. Moreover, we explore dialogue-based fine-tuning, which transforms static datasets into conversational formats to better capture iterative reasoning processes. Experiments show that dialogue-tuned models outperform traditional methods, with improvements of $9.64\%$ in multi-round reasoning scenarios and $6.18\%$ in accuracy in a noisy environment. Our findings highlight dialogue tuning as a promising approach for advancing clinically aligned and robust medical AI systems. |
| title | Dialogue is Better Than Monologue: Instructing Medical LLMs via Strategical Conversations |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2501.17860 |