Xmodel-LM Technical Report
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915025723064320 |
|---|---|
| author | Wang, Yichuan Liu, Yang Yan, Yu Wang, Qun Huang, Xucheng Jiang, Ling |
| author_facet | Wang, Yichuan Liu, Yang Yan, Yu Wang, Qun Huang, Xucheng Jiang, Ling |
| contents | We introduce Xmodel-LM, a compact and efficient 1.1B language model pre-trained on around 2 trillion tokens. Trained on our self-built dataset (Xdata), which balances Chinese and English corpora based on downstream task optimization, Xmodel-LM exhibits remarkable performance despite its smaller size. It notably surpasses existing open-source language models of similar scale. Our model checkpoints and code are publicly accessible on GitHub at https://github.com/XiaoduoAILab/XmodelLM. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_02856 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Xmodel-LM Technical Report Wang, Yichuan Liu, Yang Yan, Yu Wang, Qun Huang, Xucheng Jiang, Ling Computation and Language Artificial Intelligence We introduce Xmodel-LM, a compact and efficient 1.1B language model pre-trained on around 2 trillion tokens. Trained on our self-built dataset (Xdata), which balances Chinese and English corpora based on downstream task optimization, Xmodel-LM exhibits remarkable performance despite its smaller size. It notably surpasses existing open-source language models of similar scale. Our model checkpoints and code are publicly accessible on GitHub at https://github.com/XiaoduoAILab/XmodelLM. |
| title | Xmodel-LM Technical Report |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2406.02856 |