A Randomized Controlled Trial and Pilot of Scout: an LLM-Based EHR Search and Synthesis Platform
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914517445771264 |
|---|---|
| author | Gao, Michael Balu, Suresh Knechtle, William Pejavara, Kartik Jeck, William Ellis, Matthew Thieling, Jason Cameron, Blake Tatreau, Jason Aljurf, Tareq Foote, Henry Revoir, Michael Nichols, Marshall Gardner, Matthew Ratliff, William Hintze, Bradley Milazzo, Angelo Vemulapalli, Sreekanth |
| author_facet | Gao, Michael Balu, Suresh Knechtle, William Pejavara, Kartik Jeck, William Ellis, Matthew Thieling, Jason Cameron, Blake Tatreau, Jason Aljurf, Tareq Foote, Henry Revoir, Michael Nichols, Marshall Gardner, Matthew Ratliff, William Hintze, Bradley Milazzo, Angelo Vemulapalli, Sreekanth |
| contents | Clinical documentation and data retrieval within Electronic Health Records (EHRs) contribute substantially to clinician workload and burnout. To address this, we developed Scout, an LLM-based EHR search and synthesis platform that enables clinicians to query EHR data using natural language. Each response includes citations linking each claim to the original data source, facilitating easy verification of generated content. We conducted a prospective randomized, evaluator-blinded crossover trial across seven clinical specialties (20 participants, 200 structured cases). Participants completed realistic clinical tasks using either Scout or the EHR alone, with outcomes including time to completion, NASA Task Load Index workload scores, and blinded expert adjudication of accuracy, completeness, and relevance. Scout reduced task completion time by 37.6% and significantly decreased perceived workload, with the largest reductions in mental demand, effort, and temporal demand. Non-inferiority analyses showed that tasks completed with Scout maintained accuracy, completeness, and relevance relative to tasks completed with the EHR-only. A concurrent pilot deployment across over 200 users and more than 20 specialties generated over 6,600 interactions in three months, revealing diverse clinical and administrative use cases. Automated evaluation using an LLM-as-judge framework identified errors at low rates. Subsequent manual review of a subset of outputs revealed that most claims flagged by the automated judge as errors were in fact supported by the patient chart, demonstrating the importance of human validation. These findings provide early trial-based evidence that LLM-powered EHR tools can meaningfully reduce clinical and administrative workloads while maintaining output quality. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_26953 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | A Randomized Controlled Trial and Pilot of Scout: an LLM-Based EHR Search and Synthesis Platform Gao, Michael Balu, Suresh Knechtle, William Pejavara, Kartik Jeck, William Ellis, Matthew Thieling, Jason Cameron, Blake Tatreau, Jason Aljurf, Tareq Foote, Henry Revoir, Michael Nichols, Marshall Gardner, Matthew Ratliff, William Hintze, Bradley Milazzo, Angelo Vemulapalli, Sreekanth Information Retrieval Computers and Society Clinical documentation and data retrieval within Electronic Health Records (EHRs) contribute substantially to clinician workload and burnout. To address this, we developed Scout, an LLM-based EHR search and synthesis platform that enables clinicians to query EHR data using natural language. Each response includes citations linking each claim to the original data source, facilitating easy verification of generated content. We conducted a prospective randomized, evaluator-blinded crossover trial across seven clinical specialties (20 participants, 200 structured cases). Participants completed realistic clinical tasks using either Scout or the EHR alone, with outcomes including time to completion, NASA Task Load Index workload scores, and blinded expert adjudication of accuracy, completeness, and relevance. Scout reduced task completion time by 37.6% and significantly decreased perceived workload, with the largest reductions in mental demand, effort, and temporal demand. Non-inferiority analyses showed that tasks completed with Scout maintained accuracy, completeness, and relevance relative to tasks completed with the EHR-only. A concurrent pilot deployment across over 200 users and more than 20 specialties generated over 6,600 interactions in three months, revealing diverse clinical and administrative use cases. Automated evaluation using an LLM-as-judge framework identified errors at low rates. Subsequent manual review of a subset of outputs revealed that most claims flagged by the automated judge as errors were in fact supported by the patient chart, demonstrating the importance of human validation. These findings provide early trial-based evidence that LLM-powered EHR tools can meaningfully reduce clinical and administrative workloads while maintaining output quality. |
| title | A Randomized Controlled Trial and Pilot of Scout: an LLM-Based EHR Search and Synthesis Platform |
| topic | Information Retrieval Computers and Society |
| url | https://arxiv.org/abs/2604.26953 |