TituLLMs: A Family of Bangla LLMs with Comprehensive Benchmarking
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908395753177088 |
|---|---|
| author | Nahin, Shahriar Kabir Nandi, Rabindra Nath Sarker, Sagor Muhtaseem, Quazi Sarwar Kowsher, Md Shill, Apu Chandraw Ibrahim, Md Menon, Mehadi Hasan Muntasir, Tareq Al Alam, Firoj |
| author_facet | Nahin, Shahriar Kabir Nandi, Rabindra Nath Sarker, Sagor Muhtaseem, Quazi Sarwar Kowsher, Md Shill, Apu Chandraw Ibrahim, Md Menon, Mehadi Hasan Muntasir, Tareq Al Alam, Firoj |
| contents | In this paper, we present TituLLMs, the first large pretrained Bangla LLMs, available in 1b and 3b parameter sizes. Due to computational constraints during both training and inference, we focused on smaller models. To train TituLLMs, we collected a pretraining dataset of approximately ~37 billion tokens. We extended the Llama-3.2 tokenizer to incorporate language- and culture-specific knowledge, which also enables faster training and inference. There was a lack of benchmarking datasets to benchmark LLMs for Bangla. To address this gap, we developed five benchmarking datasets. We benchmarked various LLMs, including TituLLMs, and demonstrated that TituLLMs outperforms its initial multilingual versions. However, this is not always the case, highlighting the complexities of language adaptation. Our work lays the groundwork for adapting existing multilingual open models to other low-resource languages. To facilitate broader adoption and further research, we have made the TituLLMs models and benchmarking datasets publicly available (https://huggingface.co/collections/hishab/titulm-llama-family-6718d31fc1b83529276f490a). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_11187 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | TituLLMs: A Family of Bangla LLMs with Comprehensive Benchmarking Nahin, Shahriar Kabir Nandi, Rabindra Nath Sarker, Sagor Muhtaseem, Quazi Sarwar Kowsher, Md Shill, Apu Chandraw Ibrahim, Md Menon, Mehadi Hasan Muntasir, Tareq Al Alam, Firoj Computation and Language Artificial Intelligence 68T50 F.2.2; I.2.7 In this paper, we present TituLLMs, the first large pretrained Bangla LLMs, available in 1b and 3b parameter sizes. Due to computational constraints during both training and inference, we focused on smaller models. To train TituLLMs, we collected a pretraining dataset of approximately ~37 billion tokens. We extended the Llama-3.2 tokenizer to incorporate language- and culture-specific knowledge, which also enables faster training and inference. There was a lack of benchmarking datasets to benchmark LLMs for Bangla. To address this gap, we developed five benchmarking datasets. We benchmarked various LLMs, including TituLLMs, and demonstrated that TituLLMs outperforms its initial multilingual versions. However, this is not always the case, highlighting the complexities of language adaptation. Our work lays the groundwork for adapting existing multilingual open models to other low-resource languages. To facilitate broader adoption and further research, we have made the TituLLMs models and benchmarking datasets publicly available (https://huggingface.co/collections/hishab/titulm-llama-family-6718d31fc1b83529276f490a). |
| title | TituLLMs: A Family of Bangla LLMs with Comprehensive Benchmarking |
| topic | Computation and Language Artificial Intelligence 68T50 F.2.2; I.2.7 |
| url | https://arxiv.org/abs/2502.11187 |