Chain-of-Though (CoT) prompting strategies for medical error detection and correction
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914833688952832 |
|---|---|
| author | Wu, Zhaolong Hasan, Abul Wu, Jinge Kim, Yunsoo Cheung, Jason P. Y. Zhang, Teng Wu, Honghan |
| author_facet | Wu, Zhaolong Hasan, Abul Wu, Jinge Kim, Yunsoo Cheung, Jason P. Y. Zhang, Teng Wu, Honghan |
| contents | This paper describes our submission to the MEDIQA-CORR 2024 shared task for automatically detecting and correcting medical errors in clinical notes. We report results for three methods of few-shot In-Context Learning (ICL) augmented with Chain-of-Thought (CoT) and reason prompts using a large language model (LLM). In the first method, we manually analyse a subset of train and validation dataset to infer three CoT prompts by examining error types in the clinical notes. In the second method, we utilise the training dataset to prompt the LLM to deduce reasons about their correctness or incorrectness. The constructed CoTs and reasons are then augmented with ICL examples to solve the tasks of error detection, span identification, and error correction. Finally, we combine the two methods using a rule-based ensemble method. Across the three sub-tasks, our ensemble method achieves a ranking of 3rd for both sub-task 1 and 2, while securing 7th place in sub-task 3 among all submissions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_09103 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Chain-of-Though (CoT) prompting strategies for medical error detection and correction Wu, Zhaolong Hasan, Abul Wu, Jinge Kim, Yunsoo Cheung, Jason P. Y. Zhang, Teng Wu, Honghan Computation and Language This paper describes our submission to the MEDIQA-CORR 2024 shared task for automatically detecting and correcting medical errors in clinical notes. We report results for three methods of few-shot In-Context Learning (ICL) augmented with Chain-of-Thought (CoT) and reason prompts using a large language model (LLM). In the first method, we manually analyse a subset of train and validation dataset to infer three CoT prompts by examining error types in the clinical notes. In the second method, we utilise the training dataset to prompt the LLM to deduce reasons about their correctness or incorrectness. The constructed CoTs and reasons are then augmented with ICL examples to solve the tasks of error detection, span identification, and error correction. Finally, we combine the two methods using a rule-based ensemble method. Across the three sub-tasks, our ensemble method achieves a ranking of 3rd for both sub-task 1 and 2, while securing 7th place in sub-task 3 among all submissions. |
| title | Chain-of-Though (CoT) prompting strategies for medical error detection and correction |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2406.09103 |