Cover Image

Survey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Application

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yang, Chuanpeng, Lu, Wang, Zhu, Yao, Wang, Yidong, Chen, Qian, Gao, Chenlong, Yan, Bingjie, Chen, Yiqiang
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

A Survey on Evaluation of Large Language Models
by: Chang, Yupeng, et al.
Published: (2023)

Knowledge Editing for Large Language Models: A Survey
by: Wang, Song, et al.
Published: (2023)

KIEval: A Knowledge-grounded Interactive Evaluation Framework for Large Language Models
by: Yu, Zhuohao, et al.
Published: (2024)

EasyDistill: A Comprehensive Toolkit for Effective Knowledge Distillation of Large Language Models
by: Wang, Chengyu, et al.
Published: (2025)

FreeEval: A Modular Framework for Trustworthy and Efficient Evaluation of Large Language Models
by: Yu, Zhuohao, et al.
Published: (2024)

Dual-Space Knowledge Distillation for Large Language Models
by: Zhang, Songming, et al.
Published: (2024)

Supervised Knowledge Makes Large Language Models Better In-context Learners
by: Yang, Linyi, et al.
Published: (2023)

Large Language Model Enhanced Knowledge Representation Learning: A Survey
by: Wang, Xin, et al.
Published: (2024)

Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Models
by: Wu, Taiqiang, et al.
Published: (2024)

A Comprehensive Survey on Integrating Large Language Models with Knowledge-Based Methods
by: Yang, Wenli, et al.
Published: (2025)

Cross-Modal Knowledge Distillation for Speech Large Language Models
by: Wang, Enzhi, et al.
Published: (2025)

Contextualization Distillation from Large Language Model for Knowledge Graph Completion
by: Li, Dawei, et al.
Published: (2024)

LLM-Align: Utilizing Large Language Models for Entity Alignment in Knowledge Graphs
by: Chen, Xuan, et al.
Published: (2024)

Evaluating the Factuality of Large Language Models using Large-Scale Knowledge Graphs
by: Liu, Xiaoze, et al.
Published: (2024)

CODESYNC: Synchronizing Large Language Models with Dynamic Code Evolution at Scale
by: Wang, Chenlong, et al.
Published: (2025)

LLM-NEO: Parameter Efficient Knowledge Distillation for Large Language Models
by: Yang, Runming, et al.
Published: (2024)

Improving Long Text Understanding with Knowledge Distilled from Summarization Model
by: Liu, Yan, et al.
Published: (2024)

Towards Generalization of Block Attention via Automatic Segmentation and Block Distillation
by: Li, Shuaiyi, et al.
Published: (2026)

Large Language Models for Knowledge Graph Embedding: A Survey
by: Liu, Bingchen, et al.
Published: (2025)

A Survey on Large Language Models from General Purpose to Medical Applications: Datasets, Methodologies, and Evaluations
by: Wang, Jinqiang, et al.
Published: (2024)

AutoSurvey: Large Language Models Can Automatically Write Surveys
by: Wang, Yidong, et al.
Published: (2024)

Aligning Large Language Models with Searcher Preferences
by: Wu, Wei, et al.
Published: (2026)

KGPA: Robustness Evaluation for Large Language Models via Cross-Domain Knowledge Graphs
by: Pei, Aihua, et al.
Published: (2024)

Contrastive Learning for Knowledge-Based Question Generation in Large Language Models
by: Zhang, Zhenhong, et al.
Published: (2024)

Knowledge Distillation for Large Language Models
by: La Torre, Alejandro Paredes, et al.
Published: (2026)

Towards Context-Invariant Safety Alignment for Large Language Models
by: Wang, Yixu, et al.
Published: (2026)

Unifying Large Language Models and Knowledge Graphs: A Roadmap
by: Pan, Shirui, et al.
Published: (2023)

A Survey on Large Language Models for Mathematical Reasoning
by: Wang, Peng-Yuan, et al.
Published: (2025)

Knowledge Mechanisms in Large Language Models: A Survey and Perspective
by: Wang, Mengru, et al.
Published: (2024)

The Oscars of AI Theater: A Survey on Role-Playing with Language Models
by: Chen, Nuo, et al.
Published: (2024)

The Labyrinth and the Thread: Rethinking Regularizations in Sequential Knowledge Editing for Large Language Models
by: Wang, Zheng, et al.
Published: (2026)

Delta Knowledge Distillation for Large Language Models
by: Cao, Yihan, et al.
Published: (2025)

RoCar: A Relationship Network-based Evaluation Method for Large Language Models
by: Wang, Ming, et al.
Published: (2023)

Pedagogically-Inspired Data Synthesis for Language Model Knowledge Distillation
by: He, Bowei, et al.
Published: (2026)

Knowledge Boundary Discovery for Large Language Models
by: Wang, Ziquan, et al.
Published: (2026)

Knowledge-Driven Agentic Scientific Corpus Distillation Framework for Biomedical Large Language Models Training
by: Xiao, Meng, et al.
Published: (2025)

FedCoT: Federated Chain-of-Thought Distillation for Large Language Models
by: Fan, Tao, et al.
Published: (2024)

Distilling Event Sequence Knowledge From Large Language Models
by: Wadhwa, Somin, et al.
Published: (2024)

Two Heads Are Better Than One: Integrating Knowledge from Knowledge Graphs and Large Language Models for Entity Alignment
by: Yang, Linyao, et al.
Published: (2024)

Key-Point-Driven Mathematical Reasoning Distillation of Large Language Model
by: Zhu, Xunyu, et al.
Published: (2024)