Cover Image

Kakugo: Distillation of Low-Resource Languages into Small Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Devine, Peter, Sanni, Mardhiyah, Adilazuarda, Farid, Loizaga, Julieta Gil, Haddow, Barry
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

QLESS: A Quantized Approach for Data Valuation and Selection in Large Language Model Fine-Tuning
by: Ananta, Moses, et al.
Published: (2025)

Does Visual Rendering Bypass Tokenization? Investigating Script-Tokenizer Misalignment in Pixel-Based Language Models
by: Susanto, Lucky, et al.
Published: (2026)

Small Language Models for Privacy-Preserving Clinical Information Extraction in Low-Resource Languages
by: Ghaffarzadeh-Esfahani, Mohammadreza, et al.
Published: (2026)

SPARSEFIT: Few-shot Prompting with Sparse Fine-tuning for Jointly Generating Predictions and Natural Language Explanations
by: Solano, Jesus, et al.
Published: (2023)

Are You Sure? Rank Them Again: Repeated Ranking For Better Preference Datasets
by: Devine, Peter
Published: (2024)

Tagengo: A Multilingual Chat Dataset
by: Devine, Peter
Published: (2024)

Evaluating Fine-Tuned LLM Model For Medical Transcription With Small Low-Resource Languages Validated Dataset
by: Chowdhury, Mohammed Nowshad Ruhani, et al.
Published: (2026)

Large Multimodal Models for Low-Resource Languages: A Survey
by: Lupascu, Marian, et al.
Published: (2025)

Few-Shot Cross-Lingual Transfer for Prompting Large Language Models in Low-Resource Languages
by: Toukmaji, Christopher
Published: (2024)

NusaAksara: A Multimodal and Multilingual Benchmark for Preserving Indonesian Indigenous Scripts
by: Adilazuarda, Muhammad Farid, et al.
Published: (2025)

Emotion Classification in Low and Moderate Resource Languages
by: Tafreshi, Shabnam, et al.
Published: (2024)

Generalizable and Stable Finetuning of Pretrained Language Models on Low-Resource Texts
by: Somayajula, Sai Ashish, et al.
Published: (2024)

Contrastive Decoding for Synthetic Data Generation in Low-Resource Language Modeling
by: Ulm, Jannek, et al.
Published: (2025)

On Teacher Hacking in Language Model Distillation
by: Tiapkin, Daniil, et al.
Published: (2025)

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models
by: Raj, Bharath, et al.
Published: (2024)

A Multilingual Sentiment Lexicon for Low-Resource Language Translation using Large Languages Models and Explainable AI
by: Malinga, Melusi, et al.
Published: (2024)

LexC-Gen: Generating Data for Extremely Low-Resource Languages with Large Language Models and Bilingual Lexicons
by: Yong, Zheng-Xin, et al.
Published: (2024)

Bridging the Bosphorus: Advancing Turkish Large Language Models through Strategies for Low-Resource Language Adaptation and Benchmarking
by: Acikgoz, Emre Can, et al.
Published: (2024)

Large Language Models Explore by Latent Distilling
by: Zeng, Yuanhao, et al.
Published: (2026)

Delta Knowledge Distillation for Large Language Models
by: Cao, Yihan, et al.
Published: (2025)

Structured Agent Distillation for Large Language Model
by: Liu, Jun, et al.
Published: (2025)

Evolutionary Contrastive Distillation for Language Model Alignment
by: Katz-Samuels, Julian, et al.
Published: (2024)

TharuChat: Bootstrapping Large Language Models for a Low-Resource Language via Synthetic Data and Human Validation
by: Panth, Prajwal, et al.
Published: (2026)

Looking to Learn: Token-wise Dynamic Gating for Low-Resource Vision-Language Modelling
by: Ganescu, Bianca-Mihaela, et al.
Published: (2025)

Iterative Translation Refinement with Large Language Models
by: Chen, Pinzhen, et al.
Published: (2023)

Dependency Parsing Across the Resource Spectrum: Evaluating Architectures on High and Low-Resource Languages
by: Guan, Kevin, et al.
Published: (2026)

Investigating the Transferability of Code Repair for Low-Resource Programming Languages
by: Wong, Kyle, et al.
Published: (2024)

Stacking Small Language Models for Generalizability
by: Liang, Laurence
Published: (2024)

All Language Models Large and Small
by: Chen, Zhixun, et al.
Published: (2024)

AlignDistil: Token-Level Language Model Alignment as Adaptive Policy Distillation
by: Zhang, Songming, et al.
Published: (2025)

Language Ranker: A Metric for Quantifying LLM Performance Across High and Low-Resource Languages
by: Li, Zihao, et al.
Published: (2024)

Are Small Language Models Ready to Compete with Large Language Models for Practical Applications?
by: Sinha, Neelabh, et al.
Published: (2024)

Pruning and Distilling Mixture-of-Experts into Dense Language Models
by: Kim, Junhyuck, et al.
Published: (2026)

Distilling LLMs' Decomposition Abilities into Compact Language Models
by: Tarasov, Denis, et al.
Published: (2024)

Compact Language Models via Pruning and Knowledge Distillation
by: Muralidharan, Saurav, et al.
Published: (2024)

Towards the Law of Capacity Gap in Distilling Language Models
by: Zhang, Chen, et al.
Published: (2023)

IndiText Boost: Text Augmentation for Low Resource India Languages
by: Litake, Onkar, et al.
Published: (2024)

Plan Optimization to Bilingual Dictionary Induction for Low-Resource Language Families
by: Nasution, Arbi Haza, et al.
Published: (2020)

SynDARin: Synthesising Datasets for Automated Reasoning in Low-Resource Languages
by: Ghazaryan, Gayane, et al.
Published: (2024)

ConLID: Supervised Contrastive Learning for Low-Resource Language Identification
by: Foroutan, Negar, et al.
Published: (2025)