SinLlama -- A Large Language Model for Sinhala
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866908637191995392 |
|---|---|
| author | Aravinda, H. W. K. Sirajudeen, Rashad Karunathilake, Samith de Silva, Nisansa Ranathunga, Surangika Kaur, Rishemjit |
| author_facet | Aravinda, H. W. K. Sirajudeen, Rashad Karunathilake, Samith de Silva, Nisansa Ranathunga, Surangika Kaur, Rishemjit |
| contents | Low-resource languages such as Sinhala are often overlooked by open-source Large Language Models (LLMs). In this research, we extend an existing multilingual LLM (Llama-3-8B) to better serve Sinhala. We enhance the LLM tokenizer with Sinhala specific vocabulary and perform continual pre-training on a cleaned 10 million Sinhala corpus, resulting in the SinLlama model. This is the very first decoder-based open-source LLM with explicit Sinhala support. When SinLlama was instruction fine-tuned for three text classification tasks, it outperformed base and instruct variants of Llama-3-8B by a significant margin. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_09115 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SinLlama -- A Large Language Model for Sinhala Aravinda, H. W. K. Sirajudeen, Rashad Karunathilake, Samith de Silva, Nisansa Ranathunga, Surangika Kaur, Rishemjit Computation and Language Low-resource languages such as Sinhala are often overlooked by open-source Large Language Models (LLMs). In this research, we extend an existing multilingual LLM (Llama-3-8B) to better serve Sinhala. We enhance the LLM tokenizer with Sinhala specific vocabulary and perform continual pre-training on a cleaned 10 million Sinhala corpus, resulting in the SinLlama model. This is the very first decoder-based open-source LLM with explicit Sinhala support. When SinLlama was instruction fine-tuned for three text classification tasks, it outperformed base and instruct variants of Llama-3-8B by a significant margin. |
| title | SinLlama -- A Large Language Model for Sinhala |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2508.09115 |