RealMedQA: A pilot biomedical question answering dataset containing realistic clinical questions
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911991283580928 |
|---|---|
| author | Kell, Gregory Roberts, Angus Umansky, Serge Khare, Yuti Ahmed, Najma Patel, Nikhil Simela, Chloe Coumbe, Jack Rozario, Julian Griffiths, Ryan-Rhys Marshall, Iain J. |
| author_facet | Kell, Gregory Roberts, Angus Umansky, Serge Khare, Yuti Ahmed, Najma Patel, Nikhil Simela, Chloe Coumbe, Jack Rozario, Julian Griffiths, Ryan-Rhys Marshall, Iain J. |
| contents | Clinical question answering systems have the potential to provide clinicians with relevant and timely answers to their questions. Nonetheless, despite the advances that have been made, adoption of these systems in clinical settings has been slow. One issue is a lack of question-answering datasets which reflect the real-world needs of health professionals. In this work, we present RealMedQA, a dataset of realistic clinical questions generated by humans and an LLM. We describe the process for generating and verifying the QA pairs and assess several QA models on BioASQ and RealMedQA to assess the relative difficulty of matching answers to questions. We show that the LLM is more cost-efficient for generating "ideal" QA pairs. Additionally, we achieve a lower lexical similarity between questions and answers than BioASQ which provides an additional challenge to the top two QA models, as per the results. We release our code and our dataset publicly to encourage further research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_08624 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | RealMedQA: A pilot biomedical question answering dataset containing realistic clinical questions Kell, Gregory Roberts, Angus Umansky, Serge Khare, Yuti Ahmed, Najma Patel, Nikhil Simela, Chloe Coumbe, Jack Rozario, Julian Griffiths, Ryan-Rhys Marshall, Iain J. Computation and Language Artificial Intelligence Clinical question answering systems have the potential to provide clinicians with relevant and timely answers to their questions. Nonetheless, despite the advances that have been made, adoption of these systems in clinical settings has been slow. One issue is a lack of question-answering datasets which reflect the real-world needs of health professionals. In this work, we present RealMedQA, a dataset of realistic clinical questions generated by humans and an LLM. We describe the process for generating and verifying the QA pairs and assess several QA models on BioASQ and RealMedQA to assess the relative difficulty of matching answers to questions. We show that the LLM is more cost-efficient for generating "ideal" QA pairs. Additionally, we achieve a lower lexical similarity between questions and answers than BioASQ which provides an additional challenge to the top two QA models, as per the results. We release our code and our dataset publicly to encourage further research. |
| title | RealMedQA: A pilot biomedical question answering dataset containing realistic clinical questions |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2408.08624 |