Copertina

A Survey on Knowledge Distillation of Large Language Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Xu, Xiaohan, Li, Ming, Tao, Chongyang, Shen, Tao, Cheng, Reynold, Li, Jinyang, Xu, Can, Tao, Dacheng, Zhou, Tianyi
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Leveraging Large Language Models for NLG Evaluation: Advances and Challenges
di: Li, Zhen, et al.
Pubblicazione: (2024)

Re-Reading Improves Reasoning in Large Language Models
di: Xu, Xiaohan, et al.
Pubblicazione: (2023)

Revisiting Knowledge Distillation for Autoregressive Language Models
di: Zhong, Qihuang, et al.
Pubblicazione: (2024)

Meta-Task Prompting Elicits Embeddings from Large Language Models
di: Lei, Yibin, et al.
Pubblicazione: (2024)

Adam: Dense Retrieval Distillation with Adaptive Dark Examples
di: Tao, Chongyang, et al.
Pubblicazione: (2022)

A Survey on Self-Evolution of Large Language Models
di: Tao, Zhengwei, et al.
Pubblicazione: (2024)

Exploring and Enhancing the Transfer of Distribution in Knowledge Distillation for Autoregressive Language Models
di: Rao, Jun, et al.
Pubblicazione: (2024)

PANDA: Prompt Transfer Meets Knowledge Distillation for Efficient Model Adaptation
di: Zhong, Qihuang, et al.
Pubblicazione: (2022)

Reason-KE++: Aligning the Process, Not Just the Outcome, for Faithful LLM Knowledge Editing
di: Wu, Yuchen, et al.
Pubblicazione: (2025)

Revisiting Catastrophic Forgetting in Large Language Model Tuning
di: Li, Hongyu, et al.
Pubblicazione: (2024)

Robust Knowledge Editing via Explicit Reasoning Chains for Distractor-Resilient Multi-Hop QA
di: Wu, Yuchen, et al.
Pubblicazione: (2025)

CCPrefix: Counterfactual Contrastive Prefix-Tuning for Many-Class Classification
di: Li, Yang, et al.
Pubblicazione: (2022)

OOP: Object-Oriented Programming Evaluation Benchmark for Large Language Models
di: Wang, Shuai, et al.
Pubblicazione: (2024)

Edit Once, Update Everywhere: A Simple Framework for Cross-Lingual Knowledge Synchronization in LLMs
di: Wu, Yuchen, et al.
Pubblicazione: (2025)

Learning from Imperfect Data: Towards Efficient Knowledge Distillation of Autoregressive Language Models for Text-to-SQL
di: Zhong, Qihuang, et al.
Pubblicazione: (2024)

MMIDR: Teaching Large Language Model to Interpret Multimodal Misinformation via Knowledge Distillation
di: Wang, Longzheng, et al.
Pubblicazione: (2024)

Corpus-Steered Query Expansion with Large Language Models
di: Lei, Yibin, et al.
Pubblicazione: (2024)

Knowledge Editing for Large Language Model with Knowledge Neuronal Ensemble
di: Li, Yongchang, et al.
Pubblicazione: (2024)

Towards Distillation-Resistant Large Language Models: An Information-Theoretic Perspective
di: Fang, Hao, et al.
Pubblicazione: (2026)

Debating Truth: Debate-driven Claim Verification with Multiple Large Language Model Agents
di: He, Haorui, et al.
Pubblicazione: (2025)

LLMs are Also Effective Embedding Models: An In-depth Overview
di: Tao, Chongyang, et al.
Pubblicazione: (2024)

Large Language Models as an Indirect Reasoner: Contrapositive and Contradiction for Automated Reasoning
di: Zhang, Yanfang, et al.
Pubblicazione: (2024)

KS-LLM: Knowledge Selection of Large Language Models with Evidence Document for Question Answering
di: Zheng, Xinxin, et al.
Pubblicazione: (2024)

Direct Preference Knowledge Distillation for Large Language Models
di: Li, Yixing, et al.
Pubblicazione: (2024)

JustLogic: A Comprehensive Benchmark for Evaluating Deductive Reasoning in Large Language Models
di: Chen, Michael K., et al.
Pubblicazione: (2025)

Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Models
di: Wu, Taiqiang, et al.
Pubblicazione: (2024)

ToolCoder: A Systematic Code-Empowered Tool Learning Framework for Large Language Models
di: Ding, Hanxing, et al.
Pubblicazione: (2025)

QCRD: Quality-guided Contrastive Rationale Distillation for Large Language Models
di: Wang, Wei, et al.
Pubblicazione: (2024)

Multimodal Large Language Models Meet Multimodal Emotion Recognition and Reasoning: A Survey
di: Shou, Yuntao, et al.
Pubblicazione: (2025)

Erasing Without Remembering: Implicit Knowledge Forgetting in Large Language Models
di: Wang, Huazheng, et al.
Pubblicazione: (2025)

Exploring Knowledge Purification in Multi-Teacher Knowledge Distillation for LLMs
di: Jin, Ruihan, et al.
Pubblicazione: (2026)

Knowledge Distillation of Black-Box Large Language Models
di: Chen, Hongzhan, et al.
Pubblicazione: (2024)

WizardCoder: Empowering Code Large Language Models with Evol-Instruct
di: Luo, Ziyang, et al.
Pubblicazione: (2023)

A Survey on Agentic Multimodal Large Language Models
di: Yao, Huanjin, et al.
Pubblicazione: (2025)

A Comprehensive Evaluation on Event Reasoning of Large Language Models
di: Tao, Zhengwei, et al.
Pubblicazione: (2024)

WisdoM: Improving Multimodal Sentiment Analysis by Fusing Contextual World Knowledge
di: Wang, Wenbin, et al.
Pubblicazione: (2024)

Assessing and Mitigating Medical Knowledge Drift and Conflicts in Large Language Models
di: Wu, Weiyi, et al.
Pubblicazione: (2025)

Diversifying the Mixture-of-Experts Representation for Language Models with Orthogonal Optimizer
di: Liu, Boan, et al.
Pubblicazione: (2023)

Large Language Models for Knowledge Graph Embedding: A Survey
di: Liu, Bingchen, et al.
Pubblicazione: (2025)

Systematic Outliers in Large Language Models
di: An, Yongqi, et al.
Pubblicazione: (2025)