Few shot chain-of-thought driven reasoning to prompt LLMs for open ended medical question answering
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909350695534592 |
|---|---|
| author | Nachane, Saeel Sandeep Gramopadhye, Ojas Chanda, Prateek Ramakrishnan, Ganesh Jadhav, Kshitij Sharad Nandwani, Yatin Raghu, Dinesh Joshi, Sachindra |
| author_facet | Nachane, Saeel Sandeep Gramopadhye, Ojas Chanda, Prateek Ramakrishnan, Ganesh Jadhav, Kshitij Sharad Nandwani, Yatin Raghu, Dinesh Joshi, Sachindra |
| contents | In this paper, we propose a modified version of the MedQA-USMLE dataset, named MEDQA-OPEN, which contains open-ended medical questions without options to mimic clinical scenarios, along with clinician-approved reasoned answers. Additionally, we implement a prompt driven by Chain of Thought (CoT) reasoning, CLINICR, to mirror the prospective process of incremental reasoning, reaching a correct response to medical questions. We empirically demonstrate how CLINICR outperforms the state-of-the-art 5-shot CoT-based prompt (Liévin et al., 2022). We also present an approach that mirrors real-life clinical practice by first exploring multiple differential diagnoses through MCQ-CLINICR and subsequently narrowing down to a final diagnosis using MCQ-ELIMINATIVE. Finally, emphasizing the importance of response verification in medical settings, we utilize a reward model mechanism, replacing the elimination process performed by MCQ-ELIMINATIVE. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_04890 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Few shot chain-of-thought driven reasoning to prompt LLMs for open ended medical question answering Nachane, Saeel Sandeep Gramopadhye, Ojas Chanda, Prateek Ramakrishnan, Ganesh Jadhav, Kshitij Sharad Nandwani, Yatin Raghu, Dinesh Joshi, Sachindra Computation and Language In this paper, we propose a modified version of the MedQA-USMLE dataset, named MEDQA-OPEN, which contains open-ended medical questions without options to mimic clinical scenarios, along with clinician-approved reasoned answers. Additionally, we implement a prompt driven by Chain of Thought (CoT) reasoning, CLINICR, to mirror the prospective process of incremental reasoning, reaching a correct response to medical questions. We empirically demonstrate how CLINICR outperforms the state-of-the-art 5-shot CoT-based prompt (Liévin et al., 2022). We also present an approach that mirrors real-life clinical practice by first exploring multiple differential diagnoses through MCQ-CLINICR and subsequently narrowing down to a final diagnosis using MCQ-ELIMINATIVE. Finally, emphasizing the importance of response verification in medical settings, we utilize a reward model mechanism, replacing the elimination process performed by MCQ-ELIMINATIVE. |
| title | Few shot chain-of-thought driven reasoning to prompt LLMs for open ended medical question answering |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2403.04890 |