BitNet b1.58 2B4T Technical Report
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908337134632960 |
|---|---|
| author | Ma, Shuming Wang, Hongyu Huang, Shaohan Zhang, Xingxing Hu, Ying Song, Ting Xia, Yan Wei, Furu |
| author_facet | Ma, Shuming Wang, Hongyu Huang, Shaohan Zhang, Xingxing Hu, Ying Song, Ting Xia, Yan Wei, Furu |
| contents | We introduce BitNet b1.58 2B4T, the first open-source, native 1-bit Large Language Model (LLM) at the 2-billion parameter scale. Trained on a corpus of 4 trillion tokens, the model has been rigorously evaluated across benchmarks covering language understanding, mathematical reasoning, coding proficiency, and conversational ability. Our results demonstrate that BitNet b1.58 2B4T achieves performance on par with leading open-weight, full-precision LLMs of similar size, while offering significant advantages in computational efficiency, including substantially reduced memory footprint, energy consumption, and decoding latency. To facilitate further research and adoption, the model weights are released via Hugging Face along with open-source inference implementations for both GPU and CPU architectures. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_12285 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | BitNet b1.58 2B4T Technical Report Ma, Shuming Wang, Hongyu Huang, Shaohan Zhang, Xingxing Hu, Ying Song, Ting Xia, Yan Wei, Furu Computation and Language Machine Learning We introduce BitNet b1.58 2B4T, the first open-source, native 1-bit Large Language Model (LLM) at the 2-billion parameter scale. Trained on a corpus of 4 trillion tokens, the model has been rigorously evaluated across benchmarks covering language understanding, mathematical reasoning, coding proficiency, and conversational ability. Our results demonstrate that BitNet b1.58 2B4T achieves performance on par with leading open-weight, full-precision LLMs of similar size, while offering significant advantages in computational efficiency, including substantially reduced memory footprint, energy consumption, and decoding latency. To facilitate further research and adoption, the model weights are released via Hugging Face along with open-source inference implementations for both GPU and CPU architectures. |
| title | BitNet b1.58 2B4T Technical Report |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2504.12285 |