Cover Image

Domain Specific Specialization in Low-Resource Settings: The Efficacy of Offline Response-Based Knowledge Distillation in Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Aslan, Erdem, Erdoğmuş, Pakize
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

DSG-KD: Knowledge Distillation from Domain-Specific to General Language Models
by: Cho, Sangyeon, et al.
Published: (2024)

Dual-Space Knowledge Distillation for Large Language Models
by: Zhang, Songming, et al.
Published: (2024)

EmoDistill: Offline Emotion Skill Distillation for Language Model Agents in Adversarial Negotiation
by: Long, Yunbo, et al.
Published: (2026)

Kakugo: Distillation of Low-Resource Languages into Small Language Models
by: Devine, Peter, et al.
Published: (2026)

Cross-Modal Knowledge Distillation for Speech Large Language Models
by: Wang, Enzhi, et al.
Published: (2025)

Distilling Event Sequence Knowledge From Large Language Models
by: Wadhwa, Somin, et al.
Published: (2024)

EasyDistill: A Comprehensive Toolkit for Effective Knowledge Distillation of Large Language Models
by: Wang, Chengyu, et al.
Published: (2025)

The Rosetta Paradox: Domain-Specific Performance Inversions in Large Language Models
by: Jha, Basab, et al.
Published: (2024)

Delta Knowledge Distillation for Large Language Models
by: Cao, Yihan, et al.
Published: (2025)

O3D: Offline Data-driven Discovery and Distillation for Sequential Decision-Making with Large Language Models
by: Xiao, Yuchen, et al.
Published: (2023)

Survey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Application
by: Yang, Chuanpeng, et al.
Published: (2024)

LLMR: Knowledge Distillation with a Large Language Model-Induced Reward
by: Li, Dongheng, et al.
Published: (2024)

Contextualization Distillation from Large Language Model for Knowledge Graph Completion
by: Li, Dawei, et al.
Published: (2024)

Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Models
by: Wu, Taiqiang, et al.
Published: (2024)

Knowledge Distillation for Large Language Models
by: La Torre, Alejandro Paredes, et al.
Published: (2026)

Knowledge-tuning Large Language Models with Structured Medical Knowledge Bases for Reliable Response Generation in Chinese
by: Wang, Haochun, et al.
Published: (2023)

Continual-learning for Modelling Low-Resource Languages from Large Language Models
by: K, Santosh Srinath, et al.
Published: (2026)

Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Survey
by: Ling, Chen, et al.
Published: (2023)

PharmaGPT: Domain-Specific Large Language Models for Bio-Pharmaceutical and Chemistry
by: Chen, Linqing, et al.
Published: (2024)

Task-Specific Knowledge Distillation via Intermediate Probes
by: Brown, Ryan, et al.
Published: (2026)

Supervised Fine Tuning of Large Language Models for Domain Specific Knowledge Graph Construction:A Case Study on Hunan's Historical Celebrities
by: Hao, Junjie, et al.
Published: (2025)

Multilingual Transfer and Domain Adaptation for Low-Resource Languages of Spain
by: Luo, Yuanchang, et al.
Published: (2024)

Domain-Specific Quality Estimation for Machine Translation in Low-Resource Scenarios
by: Gurav, Namrata Patil, et al.
Published: (2026)

LegalMidm: Use-Case-Driven Legal Domain Specialization for Korean Large Language Model
by: Jang, Youngjoon, et al.
Published: (2026)

Opportunities and Challenges of Large Language Models for Low-Resource Languages in Humanities Research
by: Zhong, Tianyang, et al.
Published: (2024)

Knowledgeable Agents by Offline Reinforcement Learning from Large Language Model Rollouts
by: Pang, Jing-Cheng, et al.
Published: (2024)

Bridging Domain Knowledge and Process Discovery Using Large Language Models
by: Norouzifar, Ali, et al.
Published: (2024)

Chain-of-Rank: Enhancing Large Language Models for Domain-Specific RAG in Edge Device
by: Lee, Juntae, et al.
Published: (2025)

Survey of Specialized Large Language Model
by: Yang, Chenghan, et al.
Published: (2025)

Less is More: Selective Reflection for Compatible and Efficient Knowledge Distillation in Large Language Models
by: Liu, Lingyuan, et al.
Published: (2025)

Declarative Knowledge Distillation from Large Language Models for Visual Question Answering Datasets
by: Eiter, Thomas, et al.
Published: (2024)

Cross-Domain Content Generation with Domain-Specific Small Language Models
by: Maloo, Ankit, et al.
Published: (2024)

Hardware Phi-1.5B: A Large Language Model Encodes Hardware Domain Specific Knowledge
by: Fu, Weimin, et al.
Published: (2024)

RAFT: Adapting Language Model to Domain Specific RAG
by: Zhang, Tianjun, et al.
Published: (2024)

Multimodal Large Language Models for Low-Resource Languages: A Case Study for Basque
by: Arana, Lukas, et al.
Published: (2025)

Customized Information and Domain-centric Knowledge Graph Construction with Large Language Models
by: Wawrzik, Frank, et al.
Published: (2024)

AA-Omniscience: Evaluating Cross-Domain Knowledge Reliability in Large Language Models
by: Jackson, Declan, et al.
Published: (2025)

Can Linguistically Related Languages Guide LLM Translation in Low-Resource Settings?
by: Ramasethu, Aishwarya, et al.
Published: (2026)

SpeechLLM: Unified Speech and Language Model for Enhanced Multi-Task Understanding in Low Resource Settings
by: Yoo, Jaekwon, et al.
Published: (2025)

LLMs to Support a Domain Specific Knowledge Assistant
by: Lovin, Maria-Flavia
Published: (2025)