The Foundational Capabilities of Large Language Models in Predicting Postoperative Risks Using Clinical Notes
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908294549864448 |
|---|---|
| author | Alba, Charles Xue, Bing Abraham, Joanna Kannampallil, Thomas Lu, Chenyang |
| author_facet | Alba, Charles Xue, Bing Abraham, Joanna Kannampallil, Thomas Lu, Chenyang |
| contents | Clinical notes recorded during a patient's perioperative journey holds immense informational value. Advances in large language models (LLMs) offer opportunities for bridging this gap. Using 84,875 pre-operative notes and its associated surgical cases from 2018 to 2021, we examine the performance of LLMs in predicting six postoperative risks using various fine-tuning strategies. Pretrained LLMs outperformed traditional word embeddings by an absolute AUROC of 38.3% and AUPRC of 33.2%. Self-supervised fine-tuning further improved performance by 3.2% and 1.5%. Incorporating labels into training further increased AUROC by 1.8% and AUPRC by 2%. The highest performance was achieved with a unified foundation model, with improvements of 3.6% for AUROC and 2.6% for AUPRC compared to self-supervision, highlighting the foundational capabilities of LLMs in predicting postoperative risks, which could be potentially beneficial when deployed for perioperative care |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_17493 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | The Foundational Capabilities of Large Language Models in Predicting Postoperative Risks Using Clinical Notes Alba, Charles Xue, Bing Abraham, Joanna Kannampallil, Thomas Lu, Chenyang Computation and Language J.3; I.2.7 Clinical notes recorded during a patient's perioperative journey holds immense informational value. Advances in large language models (LLMs) offer opportunities for bridging this gap. Using 84,875 pre-operative notes and its associated surgical cases from 2018 to 2021, we examine the performance of LLMs in predicting six postoperative risks using various fine-tuning strategies. Pretrained LLMs outperformed traditional word embeddings by an absolute AUROC of 38.3% and AUPRC of 33.2%. Self-supervised fine-tuning further improved performance by 3.2% and 1.5%. Incorporating labels into training further increased AUROC by 1.8% and AUPRC by 2%. The highest performance was achieved with a unified foundation model, with improvements of 3.6% for AUROC and 2.6% for AUPRC compared to self-supervision, highlighting the foundational capabilities of LLMs in predicting postoperative risks, which could be potentially beneficial when deployed for perioperative care |
| title | The Foundational Capabilities of Large Language Models in Predicting Postoperative Risks Using Clinical Notes |
| topic | Computation and Language J.3; I.2.7 |
| url | https://arxiv.org/abs/2402.17493 |