BianCang: A Traditional Chinese Medicine Large Language Model
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915523251404800 |
|---|---|
| author | Wei, Sibo Peng, Xueping Wang, Yi-Fei Shen, Tao Si, Jiasheng Zhang, Weiyu Zhu, Fa Vasilakos, Athanasios V. Lu, Wenpeng Wu, Xiaoming Wang, Yinglong |
| author_facet | Wei, Sibo Peng, Xueping Wang, Yi-Fei Shen, Tao Si, Jiasheng Zhang, Weiyu Zhu, Fa Vasilakos, Athanasios V. Lu, Wenpeng Wu, Xiaoming Wang, Yinglong |
| contents | The surge of large language models (LLMs) has driven significant progress in medical applications, including traditional Chinese medicine (TCM). However, current medical LLMs struggle with TCM diagnosis and syndrome differentiation due to substantial differences between TCM and modern medical theory, and the scarcity of specialized, high-quality corpora. To this end, in this paper we propose BianCang, a TCM-specific LLM, using a two-stage training process that first injects domain-specific knowledge and then aligns it through targeted stimulation to enhance diagnostic and differentiation capabilities. Specifically, we constructed pre-training corpora, instruction-aligned datasets based on real hospital records, and the ChP-TCM dataset derived from the Pharmacopoeia of the People's Republic of China. We compiled extensive TCM and medical corpora for continual pre-training and supervised fine-tuning, building a comprehensive dataset to refine the model's understanding of TCM. Evaluations across 11 test sets involving 31 models and 4 tasks demonstrate the effectiveness of BianCang, offering valuable insights for future research. Code, datasets, and models are available on https://github.com/QLU-NLP/BianCang. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_11027 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | BianCang: A Traditional Chinese Medicine Large Language Model Wei, Sibo Peng, Xueping Wang, Yi-Fei Shen, Tao Si, Jiasheng Zhang, Weiyu Zhu, Fa Vasilakos, Athanasios V. Lu, Wenpeng Wu, Xiaoming Wang, Yinglong Computation and Language Artificial Intelligence The surge of large language models (LLMs) has driven significant progress in medical applications, including traditional Chinese medicine (TCM). However, current medical LLMs struggle with TCM diagnosis and syndrome differentiation due to substantial differences between TCM and modern medical theory, and the scarcity of specialized, high-quality corpora. To this end, in this paper we propose BianCang, a TCM-specific LLM, using a two-stage training process that first injects domain-specific knowledge and then aligns it through targeted stimulation to enhance diagnostic and differentiation capabilities. Specifically, we constructed pre-training corpora, instruction-aligned datasets based on real hospital records, and the ChP-TCM dataset derived from the Pharmacopoeia of the People's Republic of China. We compiled extensive TCM and medical corpora for continual pre-training and supervised fine-tuning, building a comprehensive dataset to refine the model's understanding of TCM. Evaluations across 11 test sets involving 31 models and 4 tasks demonstrate the effectiveness of BianCang, offering valuable insights for future research. Code, datasets, and models are available on https://github.com/QLU-NLP/BianCang. |
| title | BianCang: A Traditional Chinese Medicine Large Language Model |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2411.11027 |