Pareto-Optimized Open-Source LLMs for Healthcare via Context Retrieval
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913773865926656 |
|---|---|
| author | Bayarri-Planas, Jordi Gururajan, Ashwin Kumar Garcia-Gasulla, Dario |
| author_facet | Bayarri-Planas, Jordi Gururajan, Ashwin Kumar Garcia-Gasulla, Dario |
| contents | This study leverages optimized context retrieval to enhance open-source Large Language Models (LLMs) for cost-effective, high performance healthcare AI. We demonstrate that this approach achieves state-of-the-art accuracy on medical question answering at a fraction of the cost of proprietary models, significantly improving the cost-accuracy Pareto frontier on the MedQA benchmark. Key contributions include: (1) OpenMedQA, a novel benchmark revealing a performance gap in open-ended medical QA compared to multiple-choice formats; (2) a practical, reproducible pipeline for context retrieval optimization; and (3) open-source resources (Prompt Engine, CoT/ToT/Thinking databases) to empower healthcare AI development. By advancing retrieval techniques and QA evaluation, we enable more affordable and reliable LLM solutions for healthcare. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_15127 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Pareto-Optimized Open-Source LLMs for Healthcare via Context Retrieval Bayarri-Planas, Jordi Gururajan, Ashwin Kumar Garcia-Gasulla, Dario Artificial Intelligence I.2.0; I.2.7 This study leverages optimized context retrieval to enhance open-source Large Language Models (LLMs) for cost-effective, high performance healthcare AI. We demonstrate that this approach achieves state-of-the-art accuracy on medical question answering at a fraction of the cost of proprietary models, significantly improving the cost-accuracy Pareto frontier on the MedQA benchmark. Key contributions include: (1) OpenMedQA, a novel benchmark revealing a performance gap in open-ended medical QA compared to multiple-choice formats; (2) a practical, reproducible pipeline for context retrieval optimization; and (3) open-source resources (Prompt Engine, CoT/ToT/Thinking databases) to empower healthcare AI development. By advancing retrieval techniques and QA evaluation, we enable more affordable and reliable LLM solutions for healthcare. |
| title | Pareto-Optimized Open-Source LLMs for Healthcare via Context Retrieval |
| topic | Artificial Intelligence I.2.0; I.2.7 |
| url | https://arxiv.org/abs/2409.15127 |