Quantized Large Language Models in Biomedical Natural Language Processing: Evaluation and Recommendation
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911138780807168 |
|---|---|
| author | Zhan, Zaifu Zhou, Shuang Zeng, Min Yu, Kai Song, Meijia Chen, Xiaoyi Wang, Jun Hou, Yu Zhang, Rui |
| author_facet | Zhan, Zaifu Zhou, Shuang Zeng, Min Yu, Kai Song, Meijia Chen, Xiaoyi Wang, Jun Hou, Yu Zhang, Rui |
| contents | Large language models have demonstrated remarkable capabilities in biomedical natural language processing, yet their rapid growth in size and computational requirements present a major barrier to adoption in healthcare settings where data privacy precludes cloud deployment and resources are limited. In this study, we systematically evaluated the impact of quantization on 12 state-of-the-art large language models, including both general-purpose and biomedical-specific models, across eight benchmark datasets covering four key tasks: named entity recognition, relation extraction, multi-label classification, and question answering. We show that quantization substantially reduces GPU memory requirements-by up to 75%-while preserving model performance across diverse tasks, enabling the deployment of 70B-parameter models on 40GB consumer-grade GPUs. In addition, domain-specific knowledge and responsiveness to advanced prompting methods are largely maintained. These findings provide significant practical and guiding value, highlighting quantization as a practical and effective strategy for enabling the secure, local deployment of large yet high-capacity language models in biomedical contexts, bridging the gap between technical advances in AI and real-world clinical translation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_04534 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Quantized Large Language Models in Biomedical Natural Language Processing: Evaluation and Recommendation Zhan, Zaifu Zhou, Shuang Zeng, Min Yu, Kai Song, Meijia Chen, Xiaoyi Wang, Jun Hou, Yu Zhang, Rui Computation and Language Artificial Intelligence Large language models have demonstrated remarkable capabilities in biomedical natural language processing, yet their rapid growth in size and computational requirements present a major barrier to adoption in healthcare settings where data privacy precludes cloud deployment and resources are limited. In this study, we systematically evaluated the impact of quantization on 12 state-of-the-art large language models, including both general-purpose and biomedical-specific models, across eight benchmark datasets covering four key tasks: named entity recognition, relation extraction, multi-label classification, and question answering. We show that quantization substantially reduces GPU memory requirements-by up to 75%-while preserving model performance across diverse tasks, enabling the deployment of 70B-parameter models on 40GB consumer-grade GPUs. In addition, domain-specific knowledge and responsiveness to advanced prompting methods are largely maintained. These findings provide significant practical and guiding value, highlighting quantization as a practical and effective strategy for enabling the secure, local deployment of large yet high-capacity language models in biomedical contexts, bridging the gap between technical advances in AI and real-world clinical translation. |
| title | Quantized Large Language Models in Biomedical Natural Language Processing: Evaluation and Recommendation |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2509.04534 |