Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2402.13963 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916268690374656 |
|---|---|
| author | Qiu, Pengcheng Wu, Chaoyi Zhang, Xiaoman Lin, Weixiong Wang, Haicheng Zhang, Ya Wang, Yanfeng Xie, Weidi |
| author_facet | Qiu, Pengcheng Wu, Chaoyi Zhang, Xiaoman Lin, Weixiong Wang, Haicheng Zhang, Ya Wang, Yanfeng Xie, Weidi |
| contents | The development of open-source, multilingual medical language models can benefit a wide, linguistically diverse audience from different regions. To promote this domain, we present contributions from the following: First, we construct a multilingual medical corpus, containing approximately 25.5B tokens encompassing 6 main languages, termed as MMedC, enabling auto-regressive domain adaptation for general LLMs; Second, to monitor the development of multilingual medical LLMs, we propose a multilingual medical multi-choice question-answering benchmark with rationale, termed as MMedBench; Third, we have assessed a number of open-source large language models (LLMs) on our benchmark, along with those further auto-regressive trained on MMedC. Our final model, MMed-Llama 3, with only 8B parameters, achieves superior performance compared to all other open-source models on both MMedBench and English benchmarks, even rivaling GPT-4. In conclusion, in this work, we present a large-scale corpus, a benchmark and a series of models to support the development of multilingual medical LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_13963 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Towards Building Multilingual Language Model for Medicine Qiu, Pengcheng Wu, Chaoyi Zhang, Xiaoman Lin, Weixiong Wang, Haicheng Zhang, Ya Wang, Yanfeng Xie, Weidi Computation and Language The development of open-source, multilingual medical language models can benefit a wide, linguistically diverse audience from different regions. To promote this domain, we present contributions from the following: First, we construct a multilingual medical corpus, containing approximately 25.5B tokens encompassing 6 main languages, termed as MMedC, enabling auto-regressive domain adaptation for general LLMs; Second, to monitor the development of multilingual medical LLMs, we propose a multilingual medical multi-choice question-answering benchmark with rationale, termed as MMedBench; Third, we have assessed a number of open-source large language models (LLMs) on our benchmark, along with those further auto-regressive trained on MMedC. Our final model, MMed-Llama 3, with only 8B parameters, achieves superior performance compared to all other open-source models on both MMedBench and English benchmarks, even rivaling GPT-4. In conclusion, in this work, we present a large-scale corpus, a benchmark and a series of models to support the development of multilingual medical LLMs. |
| title | Towards Building Multilingual Language Model for Medicine |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2402.13963 |