MedSlice: Fine-Tuned Large Language Models for Secure Clinical Note Sectioning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913664148176896 |
|---|---|
| author | Davis, Joshua Sounack, Thomas Sciacca, Kate Brain, Jessie M Durieux, Brigitte N Agaronnik, Nicole D Lindvall, Charlotta |
| author_facet | Davis, Joshua Sounack, Thomas Sciacca, Kate Brain, Jessie M Durieux, Brigitte N Agaronnik, Nicole D Lindvall, Charlotta |
| contents | Extracting sections from clinical notes is crucial for downstream analysis but is challenging due to variability in formatting and labor-intensive nature of manual sectioning. While proprietary large language models (LLMs) have shown promise, privacy concerns limit their accessibility. This study develops a pipeline for automated note sectioning using open-source LLMs, focusing on three sections: History of Present Illness, Interval History, and Assessment and Plan. We fine-tuned three open-source LLMs to extract sections using a curated dataset of 487 progress notes, comparing results relative to proprietary models (GPT-4o, GPT-4o mini). Internal and external validity were assessed via precision, recall and F1 score. Fine-tuned Llama 3.1 8B outperformed GPT-4o (F1=0.92). On the external validity test set, performance remained high (F1= 0.85). Fine-tuned open-source LLMs can surpass proprietary models in clinical note sectioning, offering advantages in cost, performance, and accessibility. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_14105 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MedSlice: Fine-Tuned Large Language Models for Secure Clinical Note Sectioning Davis, Joshua Sounack, Thomas Sciacca, Kate Brain, Jessie M Durieux, Brigitte N Agaronnik, Nicole D Lindvall, Charlotta Computation and Language Artificial Intelligence Information Retrieval Machine Learning Extracting sections from clinical notes is crucial for downstream analysis but is challenging due to variability in formatting and labor-intensive nature of manual sectioning. While proprietary large language models (LLMs) have shown promise, privacy concerns limit their accessibility. This study develops a pipeline for automated note sectioning using open-source LLMs, focusing on three sections: History of Present Illness, Interval History, and Assessment and Plan. We fine-tuned three open-source LLMs to extract sections using a curated dataset of 487 progress notes, comparing results relative to proprietary models (GPT-4o, GPT-4o mini). Internal and external validity were assessed via precision, recall and F1 score. Fine-tuned Llama 3.1 8B outperformed GPT-4o (F1=0.92). On the external validity test set, performance remained high (F1= 0.85). Fine-tuned open-source LLMs can surpass proprietary models in clinical note sectioning, offering advantages in cost, performance, and accessibility. |
| title | MedSlice: Fine-Tuned Large Language Models for Secure Clinical Note Sectioning |
| topic | Computation and Language Artificial Intelligence Information Retrieval Machine Learning |
| url | https://arxiv.org/abs/2501.14105 |