Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2412.05845 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917861441667072 |
|---|---|
| author | Li, Yahan Harrigian, Keith Zirikly, Ayah Dredze, Mark |
| author_facet | Li, Yahan Harrigian, Keith Zirikly, Ayah Dredze, Mark |
| contents | Large language models with a transformer-based encoder/decoder architecture, such as T5, have become standard platforms for supervised tasks. To bring these technologies to the clinical domain, recent work has trained new or adapted existing models to clinical data. However, the evaluation of these clinical T5 models and comparison to other models has been limited. Are the clinical T5 models better choices than FLAN-tuned generic T5 models? Do they generalize better to new clinical domains that differ from the training sets? We comprehensively evaluate these models across several clinical tasks and domains. We find that clinical T5 models provide marginal improvements over existing models, and perform worse when evaluated on different domains. Our results inform future choices in developing clinical LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_05845 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Are Clinical T5 Models Better for Clinical Text? Li, Yahan Harrigian, Keith Zirikly, Ayah Dredze, Mark Computation and Language Large language models with a transformer-based encoder/decoder architecture, such as T5, have become standard platforms for supervised tasks. To bring these technologies to the clinical domain, recent work has trained new or adapted existing models to clinical data. However, the evaluation of these clinical T5 models and comparison to other models has been limited. Are the clinical T5 models better choices than FLAN-tuned generic T5 models? Do they generalize better to new clinical domains that differ from the training sets? We comprehensively evaluate these models across several clinical tasks and domains. We find that clinical T5 models provide marginal improvements over existing models, and perform worse when evaluated on different domains. Our results inform future choices in developing clinical LLMs. |
| title | Are Clinical T5 Models Better for Clinical Text? |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2412.05845 |