Kakugo: Distillation of Low-Resource Languages into Small Language Models
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866912835156574208 |
|---|---|
| author | Devine, Peter Sanni, Mardhiyah Adilazuarda, Farid Loizaga, Julieta Gil Haddow, Barry |
| author_facet | Devine, Peter Sanni, Mardhiyah Adilazuarda, Farid Loizaga, Julieta Gil Haddow, Barry |
| contents | We present Kakugo, a novel and cost-effective pipeline designed to train general-purpose Small Language Models (SLMs) for low-resource languages using only the language name as input. By using a large teacher model to generate synthetic prompts and translate instruction datasets, we produced training data and SLMs for 54 low-resource languages. Evaluations across a diverse set of general natural language processing tasks, including translation, classification, and question answering, demonstrate that our pipeline consistently improves performance over base models. With a total generation and training cost of under $50 per language, Kakugo offers an accessible method for communities to develop language-specific AI. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_14051 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Kakugo: Distillation of Low-Resource Languages into Small Language Models Devine, Peter Sanni, Mardhiyah Adilazuarda, Farid Loizaga, Julieta Gil Haddow, Barry Computation and Language Artificial Intelligence Machine Learning We present Kakugo, a novel and cost-effective pipeline designed to train general-purpose Small Language Models (SLMs) for low-resource languages using only the language name as input. By using a large teacher model to generate synthetic prompts and translate instruction datasets, we produced training data and SLMs for 54 low-resource languages. Evaluations across a diverse set of general natural language processing tasks, including translation, classification, and question answering, demonstrate that our pipeline consistently improves performance over base models. With a total generation and training cost of under $50 per language, Kakugo offers an accessible method for communities to develop language-specific AI. |
| title | Kakugo: Distillation of Low-Resource Languages into Small Language Models |
| topic | Computation and Language Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2601.14051 |