Cover Image

Evolving Knowledge Distillation with Large Language Models and Active Learning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Chengyuan, Kang, Yangyang, Zhao, Fubang, Kuang, Kun, Jiang, Zhuoren, Sun, Changlong, Wu, Fei
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

More Than Catastrophic Forgetting: Integrating General Capabilities For Domain-Specific LLMs
by: Liu, Chengyuan, et al.
Published: (2024)

RexUniNLU: Recursive Method with Explicit Schema Instructor for Universal NLU
by: Liu, Chengyuan, et al.
Published: (2024)

Gold Panning in Vocabulary: An Adaptive Method for Vocabulary Expansion of Domain-Specific LLMs
by: Liu, Chengyuan, et al.
Published: (2024)

LLM-based Medical Assistant Personalization with Short- and Long-Term Memory Coordination
by: Zhang, Kai, et al.
Published: (2023)

Can Large Language Models Grasp Legal Theories? Enhance Legal Reasoning with Insights from Multi-Agent Collaboration
by: Yuan, Weikang, et al.
Published: (2024)

Learning to Solve Domain-Specific Calculation Problems with Knowledge-Intensive Programs Generator
by: Liu, Chengyuan, et al.
Published: (2024)

P2S: Probabilistic Process Supervision for General-Domain Reasoning Question Answering
by: Zhong, Wenlin, et al.
Published: (2026)

PoliLegalLM: A Technical Report on a Large Language Model for Political and Legal Affairs
by: Huang, Yuting, et al.
Published: (2026)

Causal Agent based on Large Language Model
by: Han, Kairong, et al.
Published: (2024)

Delta Knowledge Distillation for Large Language Models
by: Cao, Yihan, et al.
Published: (2025)

Leveraging Print Debugging to Improve Code Generation in Large Language Models
by: Hu, Xueyu, et al.
Published: (2024)

Fine-tuning Large Language Models for Improving Factuality in Legal Question Answering
by: Hu, Yinghao, et al.
Published: (2025)

Rewrite to Jailbreak: Discover Learnable and Transferable Implicit Harmfulness Instruction
by: Huang, Yuting, et al.
Published: (2025)

SWITCH: Studying with Teacher for Knowledge Distillation of Large Language Models
by: Koo, Jahyun, et al.
Published: (2024)

Enhance Robustness of Language Models Against Variation Attack through Graph Integration
by: Xiong, Zi, et al.
Published: (2024)

A Speaker Turn-Aware Multi-Task Adversarial Network for Joint User Satisfaction Estimation and Sentiment Analysis
by: Song, Kaisong, et al.
Published: (2024)

Knowledge Distillation for Temporal Knowledge Graph Reasoning with Large Language Models
by: Xing, Wang, et al.
Published: (2026)

DDK: Distilling Domain Knowledge for Efficient Large Language Models
by: Liu, Jiaheng, et al.
Published: (2024)

AppealCase: A Dataset and Benchmark for Civil Case Appeal Scenarios
by: Huang, Yuting, et al.
Published: (2025)

WisdomInterrogatory (LuWen): An Open-Source Legal Large Language Model Technical Report
by: Wu, Yiquan, et al.
Published: (2026)

CAT: Causal Attention Tuning For Injecting Fine-grained Causal Knowledge into Large Language Models
by: Han, Kairong, et al.
Published: (2025)

Dual-Space Knowledge Distillation for Large Language Models
by: Zhang, Songming, et al.
Published: (2024)

ELAD: Explanation-Guided Large Language Models Active Distillation
by: Zhang, Yifei, et al.
Published: (2024)

Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Models
by: Wu, Taiqiang, et al.
Published: (2024)

When Large Multimodal Models Confront Evolving Knowledge: Challenges and Explorations
by: Jiang, Kailin, et al.
Published: (2025)

Towards Stepwise Domain Knowledge-Driven Reasoning Optimization and Reflection Improvement
by: Liu, Chengyuan, et al.
Published: (2025)

C$^2$DLM: Causal Concept-Guided Diffusion Large Language Models
by: Han, Kairong, et al.
Published: (2025)

Knowledge in Superposition: Unveiling the Failures of Lifelong Knowledge Editing for Large Language Models
by: Hu, Chenhui, et al.
Published: (2024)

RKLD: Reverse KL-Divergence-based Knowledge Distillation for Unlearning Personal Information in Large Language Models
by: Wang, Bichen, et al.
Published: (2024)

Causality for Large Language Models
by: Wu, Anpeng, et al.
Published: (2024)

Learning Evolving Tools for Large Language Models
by: Chen, Guoxin, et al.
Published: (2024)

Knowledge Distillation of Black-Box Large Language Models
by: Chen, Hongzhan, et al.
Published: (2024)

Direct Preference Knowledge Distillation for Large Language Models
by: Li, Yixing, et al.
Published: (2024)

A Survey on Knowledge Distillation of Large Language Models
by: Xu, Xiaohan, et al.
Published: (2024)

Distilling Rule-based Knowledge into Large Language Models
by: Yang, Wenkai, et al.
Published: (2023)

Catching Chameleons: Detecting Evolving Disinformation Generated using Large Language Models
by: Jiang, Bohan, et al.
Published: (2024)

Knowledge Distillation and Dataset Distillation of Large Language Models: Emerging Trends, Challenges, and Future Directions
by: Fang, Luyang, et al.
Published: (2025)

Does Knowledge Distillation Matter for Large Language Model based Bundle Generation?
by: Feng, Kaidong, et al.
Published: (2025)

Evolving Knowledge Distillation for Lightweight Neural Machine Translation
by: Zhang, Xuewen, et al.
Published: (2026)

CogMG: Collaborative Augmentation Between Large Language Model and Knowledge Graph
by: Zhou, Tong, et al.
Published: (2024)