Cover Image

A Survey on Knowledge Distillation of Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Xu, Xiaohan, Li, Ming, Tao, Chongyang, Shen, Tao, Cheng, Reynold, Li, Jinyang, Xu, Can, Tao, Dacheng, Zhou, Tianyi
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Leveraging Large Language Models for NLG Evaluation: Advances and Challenges
by: Li, Zhen, et al.
Published: (2024)

Re-Reading Improves Reasoning in Large Language Models
by: Xu, Xiaohan, et al.
Published: (2023)

Revisiting Knowledge Distillation for Autoregressive Language Models
by: Zhong, Qihuang, et al.
Published: (2024)

Meta-Task Prompting Elicits Embeddings from Large Language Models
by: Lei, Yibin, et al.
Published: (2024)

Adam: Dense Retrieval Distillation with Adaptive Dark Examples
by: Tao, Chongyang, et al.
Published: (2022)

A Survey on Self-Evolution of Large Language Models
by: Tao, Zhengwei, et al.
Published: (2024)

Exploring and Enhancing the Transfer of Distribution in Knowledge Distillation for Autoregressive Language Models
by: Rao, Jun, et al.
Published: (2024)

PANDA: Prompt Transfer Meets Knowledge Distillation for Efficient Model Adaptation
by: Zhong, Qihuang, et al.
Published: (2022)

Reason-KE++: Aligning the Process, Not Just the Outcome, for Faithful LLM Knowledge Editing
by: Wu, Yuchen, et al.
Published: (2025)

Revisiting Catastrophic Forgetting in Large Language Model Tuning
by: Li, Hongyu, et al.
Published: (2024)

Robust Knowledge Editing via Explicit Reasoning Chains for Distractor-Resilient Multi-Hop QA
by: Wu, Yuchen, et al.
Published: (2025)

CCPrefix: Counterfactual Contrastive Prefix-Tuning for Many-Class Classification
by: Li, Yang, et al.
Published: (2022)

OOP: Object-Oriented Programming Evaluation Benchmark for Large Language Models
by: Wang, Shuai, et al.
Published: (2024)

Edit Once, Update Everywhere: A Simple Framework for Cross-Lingual Knowledge Synchronization in LLMs
by: Wu, Yuchen, et al.
Published: (2025)

Learning from Imperfect Data: Towards Efficient Knowledge Distillation of Autoregressive Language Models for Text-to-SQL
by: Zhong, Qihuang, et al.
Published: (2024)

MMIDR: Teaching Large Language Model to Interpret Multimodal Misinformation via Knowledge Distillation
by: Wang, Longzheng, et al.
Published: (2024)

Corpus-Steered Query Expansion with Large Language Models
by: Lei, Yibin, et al.
Published: (2024)

Knowledge Editing for Large Language Model with Knowledge Neuronal Ensemble
by: Li, Yongchang, et al.
Published: (2024)

Towards Distillation-Resistant Large Language Models: An Information-Theoretic Perspective
by: Fang, Hao, et al.
Published: (2026)

Debating Truth: Debate-driven Claim Verification with Multiple Large Language Model Agents
by: He, Haorui, et al.
Published: (2025)

LLMs are Also Effective Embedding Models: An In-depth Overview
by: Tao, Chongyang, et al.
Published: (2024)

Large Language Models as an Indirect Reasoner: Contrapositive and Contradiction for Automated Reasoning
by: Zhang, Yanfang, et al.
Published: (2024)

KS-LLM: Knowledge Selection of Large Language Models with Evidence Document for Question Answering
by: Zheng, Xinxin, et al.
Published: (2024)

Direct Preference Knowledge Distillation for Large Language Models
by: Li, Yixing, et al.
Published: (2024)

JustLogic: A Comprehensive Benchmark for Evaluating Deductive Reasoning in Large Language Models
by: Chen, Michael K., et al.
Published: (2025)

Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Models
by: Wu, Taiqiang, et al.
Published: (2024)

ToolCoder: A Systematic Code-Empowered Tool Learning Framework for Large Language Models
by: Ding, Hanxing, et al.
Published: (2025)

QCRD: Quality-guided Contrastive Rationale Distillation for Large Language Models
by: Wang, Wei, et al.
Published: (2024)

Erasing Without Remembering: Implicit Knowledge Forgetting in Large Language Models
by: Wang, Huazheng, et al.
Published: (2025)

Multimodal Large Language Models Meet Multimodal Emotion Recognition and Reasoning: A Survey
by: Shou, Yuntao, et al.
Published: (2025)

Exploring Knowledge Purification in Multi-Teacher Knowledge Distillation for LLMs
by: Jin, Ruihan, et al.
Published: (2026)

Knowledge Distillation of Black-Box Large Language Models
by: Chen, Hongzhan, et al.
Published: (2024)

WizardCoder: Empowering Code Large Language Models with Evol-Instruct
by: Luo, Ziyang, et al.
Published: (2023)

A Survey on Agentic Multimodal Large Language Models
by: Yao, Huanjin, et al.
Published: (2025)

A Comprehensive Evaluation on Event Reasoning of Large Language Models
by: Tao, Zhengwei, et al.
Published: (2024)

WisdoM: Improving Multimodal Sentiment Analysis by Fusing Contextual World Knowledge
by: Wang, Wenbin, et al.
Published: (2024)

Assessing and Mitigating Medical Knowledge Drift and Conflicts in Large Language Models
by: Wu, Weiyi, et al.
Published: (2025)

Diversifying the Mixture-of-Experts Representation for Language Models with Orthogonal Optimizer
by: Liu, Boan, et al.
Published: (2023)

Large Language Models for Knowledge Graph Embedding: A Survey
by: Liu, Bingchen, et al.
Published: (2025)

Systematic Outliers in Large Language Models
by: An, Yongqi, et al.
Published: (2025)