MKRAG: Medical Knowledge Retrieval Augmented Generation for Medical Question Answering
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916359264272384 |
|---|---|
| author | Shi, Yucheng Xu, Shaochen Yang, Tianze Liu, Zhengliang Liu, Tianming Li, Quanzheng Li, Xiang Liu, Ninghao |
| author_facet | Shi, Yucheng Xu, Shaochen Yang, Tianze Liu, Zhengliang Liu, Tianming Li, Quanzheng Li, Xiang Liu, Ninghao |
| contents | Large Language Models (LLMs), although powerful in general domains, often perform poorly on domain-specific tasks such as medical question answering (QA). In addition, LLMs tend to function as "black-boxes", making it challenging to modify their behavior. To address the problem, our work employs a transparent process of retrieval augmented generation (RAG), aiming to improve LLM responses without the need for fine-tuning or retraining. Specifically, we propose a comprehensive retrieval strategy to extract medical facts from an external knowledge base, and then inject them into the LLM's query prompt. Focusing on medical QA, we evaluate the impact of different retrieval models and the number of facts on LLM performance using the MedQA-SMILE dataset. Notably, our retrieval-augmented Vicuna-7B model exhibited an accuracy improvement from 44.46% to 48.54%. This work underscores the potential of RAG to enhance LLM performance, offering a practical approach to mitigate the challenges posed by black-box LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_16035 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | MKRAG: Medical Knowledge Retrieval Augmented Generation for Medical Question Answering Shi, Yucheng Xu, Shaochen Yang, Tianze Liu, Zhengliang Liu, Tianming Li, Quanzheng Li, Xiang Liu, Ninghao Computation and Language Artificial Intelligence Large Language Models (LLMs), although powerful in general domains, often perform poorly on domain-specific tasks such as medical question answering (QA). In addition, LLMs tend to function as "black-boxes", making it challenging to modify their behavior. To address the problem, our work employs a transparent process of retrieval augmented generation (RAG), aiming to improve LLM responses without the need for fine-tuning or retraining. Specifically, we propose a comprehensive retrieval strategy to extract medical facts from an external knowledge base, and then inject them into the LLM's query prompt. Focusing on medical QA, we evaluate the impact of different retrieval models and the number of facts on LLM performance using the MedQA-SMILE dataset. Notably, our retrieval-augmented Vicuna-7B model exhibited an accuracy improvement from 44.46% to 48.54%. This work underscores the potential of RAG to enhance LLM performance, offering a practical approach to mitigate the challenges posed by black-box LLMs. |
| title | MKRAG: Medical Knowledge Retrieval Augmented Generation for Medical Question Answering |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2309.16035 |