Imagen de Portada

DSG-KD: Knowledge Distillation from Domain-Specific to General Language Models

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Cho, Sangyeon, Jeon, Jangyeong, Lee, Dongjoon, Lee, Changhee, Kim, Junyeong
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

BioBridge: Unified Bio-Embedding with Bridging Modality in Code-Switched EMR
por: Jeon, Jangyeong, et al.
Publicado: (2024)

ConCSE: Unified Contrastive Learning and Augmentation for Code-Switched Embeddings
por: Jeon, Jangyeong, et al.
Publicado: (2024)

PromptKD: Distilling Student-Friendly Knowledge for Generative Language Models via Prompt Tuning
por: Kim, Gyeongman, et al.
Publicado: (2024)

Mentor-KD: Making Small Language Models Better Multi-step Reasoners
por: Lee, Hojae, et al.
Publicado: (2024)

STEPER: Step-wise Knowledge Distillation for Enhancing Reasoning Ability in Multi-Step Retrieval-Augmented Language Models
por: Lee, Kyumin, et al.
Publicado: (2025)

Synergy-CLIP: Extending CLIP with Multi-modal Integration for Robust Representation Learning
por: Cho, Sangyeon, et al.
Publicado: (2025)

A2D: Any-Order, Any-Step Safety Alignment for Diffusion Language Models
por: Jeung, Wonje, et al.
Publicado: (2025)

Ontology-Free General-Domain Knowledge Graph-to-Text Generation Dataset Synthesis using Large Language Model
por: Kim, Daehee, et al.
Publicado: (2024)

K-COMP: Retrieval-Augmented Medical Domain Question Answering With Knowledge-Injected Compressor
por: Cho, Jeonghun, et al.
Publicado: (2025)

Domain Specific Specialization in Low-Resource Settings: The Efficacy of Offline Response-Based Knowledge Distillation in Large Language Models
por: Aslan, Erdem, et al.
Publicado: (2026)

Building Domain-Specific Small Language Models via Guided Data Generation
por: Kumar, Aman, et al.
Publicado: (2025)

Knowledge Entropy Decay during Language Model Pretraining Hinders New Knowledge Acquisition
por: Kim, Jiyeon, et al.
Publicado: (2024)

ChroKnowledge: Unveiling Chronological Knowledge of Language Models in Multiple Domains
por: Park, Yein, et al.
Publicado: (2024)

Cross-Domain Content Generation with Domain-Specific Small Language Models
por: Maloo, Ankit, et al.
Publicado: (2024)

Chain-of-Rank: Enhancing Large Language Models for Domain-Specific RAG in Edge Device
por: Lee, Juntae, et al.
Publicado: (2025)

Latent Paraphrasing: Perturbation on Layers Improves Knowledge Injection in Language Models
por: Kang, Minki, et al.
Publicado: (2024)

Retrieval-Augmented Fine-Tuning With Preference Optimization For Visual Program Generation
por: Kang, Deokhyung, et al.
Publicado: (2025)

Incorporating Domain Knowledge into Materials Tokenization
por: Oh, Yerim, et al.
Publicado: (2025)

Large Language Models as Planning Domain Generators
por: Oswald, James, et al.
Publicado: (2024)

SLM as Guardian: Pioneering AI Safety with Small Language Models
por: Kwon, Ohjoon, et al.
Publicado: (2024)

Personalizing Large Language Models using Retrieval Augmented Generation and Knowledge Graph
por: Prahlad, Deeksha, et al.
Publicado: (2025)

FineScope : SAE-guided Data Selection Enables Domain Specific LLM Pruning and Finetuning
por: Bhattacharyya, Chaitali, et al.
Publicado: (2025)

A Multi-faceted Analysis of Cognitive Abilities: Evaluating Prompt Methods with Large Language Models on the CONSORT Checklist
por: Jeon, Sohyeon, et al.
Publicado: (2025)

Simplifying Multimodality: Unimodal Approach to Multimodal Challenges in Radiology with General-Domain Large Language Model
por: Cho, Seonhee, et al.
Publicado: (2024)

PCEval: A Benchmark for Evaluating Physical Computing Capabilities of Large Language Models
por: Song, Inpyo, et al.
Publicado: (2025)

Accelerating Multilingual Language Model for Excessively Tokenized Languages
por: Hong, Jimin, et al.
Publicado: (2024)

Task-Specific Knowledge Distillation via Intermediate Probes
por: Brown, Ryan, et al.
Publicado: (2026)

Knowledge Synthesis of Photosynthesis Research Using a Large Language Model
por: Yoon, Seungri, et al.
Publicado: (2025)

Pruning and Distilling Mixture-of-Experts into Dense Language Models
por: Kim, Junhyuck, et al.
Publicado: (2026)

Open Ko-LLM Leaderboard: Evaluating Large Language Models in Korean with Ko-H5 Benchmark
por: Park, Chanjun, et al.
Publicado: (2024)

GFlowPO: Generative Flow Network as a Language Model Prompt Optimizer
por: Cho, Junmo, et al.
Publicado: (2026)

MapCoder-Lite: Distilling Multi-Agent Coding into a Single Small LLM
por: Lee, Woongkyu, et al.
Publicado: (2025)

Distilling LLM Agent into Small Models with Retrieval and Code Tools
por: Kang, Minki, et al.
Publicado: (2025)

Self-Knowledge Distillation for Learning Ambiguity
por: Park, Hancheol, et al.
Publicado: (2024)

Thunder-Tok: Minimizing Tokens per Word in Tokenizing Korean Texts for Generative Language Models
por: Cho, Gyeongje, et al.
Publicado: (2025)

WAND: Windowed Attention and Knowledge Distillation for Efficient Autoregressive Text-to-Speech Models
por: Lee, Hanna, et al.
Publicado: (2026)

STEAM: A Semantic-Level Knowledge Editing Framework for Large Language Models
por: Jeong, Geunyeong, et al.
Publicado: (2025)

A Dual-Prompting for Interpretable Mental Health Language Models
por: Jeon, Hyolim, et al.
Publicado: (2024)

PICKT: Practical Interlinked Concept Knowledge Tracing for Personalized Learning using Knowledge Map Concept Relations
por: Lee, Wonbeen, et al.
Publicado: (2025)

RAFT: Adapting Language Model to Domain Specific RAG
por: Zhang, Tianjun, et al.
Publicado: (2024)