Improving Generalizability of Extracting Social Determinants of Health Using Large Language Models through Prompt-tuning
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909141587460096 |
|---|---|
| author | Peng, Cheng Yu, Zehao Smith, Kaleb E Lo-Ciganic, Wei-Hsuan Bian, Jiang Wu, Yonghui |
| author_facet | Peng, Cheng Yu, Zehao Smith, Kaleb E Lo-Ciganic, Wei-Hsuan Bian, Jiang Wu, Yonghui |
| contents | The progress in natural language processing (NLP) using large language models (LLMs) has greatly improved patient information extraction from clinical narratives. However, most methods based on the fine-tuning strategy have limited transfer learning ability for cross-domain applications. This study proposed a novel approach that employs a soft prompt-based learning architecture, which introduces trainable prompts to guide LLMs toward desired outputs. We examined two types of LLM architectures, including encoder-only GatorTron and decoder-only GatorTronGPT, and evaluated their performance for the extraction of social determinants of health (SDoH) using a cross-institution dataset from the 2022 n2c2 challenge and a cross-disease dataset from the University of Florida (UF) Health. The results show that decoder-only LLMs with prompt tuning achieved better performance in cross-domain applications. GatorTronGPT achieved the best F1 scores for both datasets, outperforming traditional fine-tuned GatorTron by 8.9% and 21.8% in a cross-institution setting, and 5.5% and 14.5% in a cross-disease setting. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_12374 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Improving Generalizability of Extracting Social Determinants of Health Using Large Language Models through Prompt-tuning Peng, Cheng Yu, Zehao Smith, Kaleb E Lo-Ciganic, Wei-Hsuan Bian, Jiang Wu, Yonghui Computation and Language The progress in natural language processing (NLP) using large language models (LLMs) has greatly improved patient information extraction from clinical narratives. However, most methods based on the fine-tuning strategy have limited transfer learning ability for cross-domain applications. This study proposed a novel approach that employs a soft prompt-based learning architecture, which introduces trainable prompts to guide LLMs toward desired outputs. We examined two types of LLM architectures, including encoder-only GatorTron and decoder-only GatorTronGPT, and evaluated their performance for the extraction of social determinants of health (SDoH) using a cross-institution dataset from the 2022 n2c2 challenge and a cross-disease dataset from the University of Florida (UF) Health. The results show that decoder-only LLMs with prompt tuning achieved better performance in cross-domain applications. GatorTronGPT achieved the best F1 scores for both datasets, outperforming traditional fine-tuned GatorTron by 8.9% and 21.8% in a cross-institution setting, and 5.5% and 14.5% in a cross-disease setting. |
| title | Improving Generalizability of Extracting Social Determinants of Health Using Large Language Models through Prompt-tuning |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2403.12374 |