Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2503.17550 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909547799511040 |
|---|---|
| author | Kumar, Niranjan Seifi, Farid Conte, Marisa Flynn, Allen |
| author_facet | Kumar, Niranjan Seifi, Farid Conte, Marisa Flynn, Allen |
| contents | Clinical calculators are widely used, and large language models (LLMs) make it possible to engage them using natural language. We demonstrate a purpose-built chatbot that leverages software implementations of verifiable clinical calculators via LLM tools and metadata about these calculators via retrieval augmented generation (RAG). We compare the chatbot's response accuracy to an unassisted off-the-shelf LLM on four natural language conversation workloads. Our chatbot achieves 100% accuracy on queries interrogating calculator metadata content and shows a significant increase in clinical calculation accuracy vs. the off-the-shelf LLM when prompted with complete sentences (86.4% vs. 61.8%) or with medical shorthand (79.2% vs. 62.0%). It eliminates calculation errors when prompted with complete sentences (0% vs. 16.8%) and greatly reduces them when prompted with medical shorthand (2.4% vs. 18%). While our chatbot is not ready for clinical use, these results show progress in minimizing incorrect calculation results. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_17550 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | An LLM-Powered Clinical Calculator Chatbot Backed by Verifiable Clinical Calculators and their Metadata Kumar, Niranjan Seifi, Farid Conte, Marisa Flynn, Allen Quantitative Methods Clinical calculators are widely used, and large language models (LLMs) make it possible to engage them using natural language. We demonstrate a purpose-built chatbot that leverages software implementations of verifiable clinical calculators via LLM tools and metadata about these calculators via retrieval augmented generation (RAG). We compare the chatbot's response accuracy to an unassisted off-the-shelf LLM on four natural language conversation workloads. Our chatbot achieves 100% accuracy on queries interrogating calculator metadata content and shows a significant increase in clinical calculation accuracy vs. the off-the-shelf LLM when prompted with complete sentences (86.4% vs. 61.8%) or with medical shorthand (79.2% vs. 62.0%). It eliminates calculation errors when prompted with complete sentences (0% vs. 16.8%) and greatly reduces them when prompted with medical shorthand (2.4% vs. 18%). While our chatbot is not ready for clinical use, these results show progress in minimizing incorrect calculation results. |
| title | An LLM-Powered Clinical Calculator Chatbot Backed by Verifiable Clinical Calculators and their Metadata |
| topic | Quantitative Methods |
| url | https://arxiv.org/abs/2503.17550 |