Cover Image

Pedagogically-Inspired Data Synthesis for Language Model Knowledge Distillation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	He, Bowei, Chen, Yankai, Zhang, Xiaokun, Kong, Linghe, Yu, Philip S., Liu, Xue, Ma, Chen
Format:	Preprint
Published:	2026
Subjects:	Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Reasoning in Action: MCTS-Driven Knowledge Retrieval for Large Language Models
by: Liu, Shuqi, et al.
Published: (2025)

Dual-Space Knowledge Distillation for Large Language Models
by: Zhang, Songming, et al.
Published: (2024)

Benchmarking the Pedagogical Knowledge of Large Language Models
by: Lelièvre, Maxime, et al.
Published: (2025)

Search-R2: Enhancing Search-Integrated Reasoning via Actor-Refiner Collaboration
by: He, Bowei, et al.
Published: (2026)

Bi-Chainer: Automated Large Language Models Reasoning with Bidirectional Chaining
by: Liu, Shuqi, et al.
Published: (2024)

Contextualization Distillation from Large Language Model for Knowledge Graph Completion
by: Li, Dawei, et al.
Published: (2024)

KDFlow: A User-Friendly and Efficient Knowledge Distillation Framework for Large Language Models
by: Zhang, Songming, et al.
Published: (2026)

GIVE: Structured Reasoning of Large Language Models with Knowledge Graph Inspired Veracity Extrapolation
by: He, Jiashu, et al.
Published: (2024)

A Dual-Space Framework for General Knowledge Distillation of Large Language Models
by: Zhang, Xue, et al.
Published: (2025)

LightReasoner: Can Small Language Models Teach Large Language Models Reasoning?
by: Wang, Jingyuan, et al.
Published: (2025)

Survey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Application
by: Yang, Chuanpeng, et al.
Published: (2024)

Less is More: Selective Reflection for Compatible and Efficient Knowledge Distillation in Large Language Models
by: Liu, Lingyuan, et al.
Published: (2025)

AlignDistil: Token-Level Language Model Alignment as Adaptive Policy Distillation
by: Zhang, Songming, et al.
Published: (2025)

Improving Long Text Understanding with Knowledge Distilled from Summarization Model
by: Liu, Yan, et al.
Published: (2024)

Teaching-Inspired Integrated Prompting Framework: A Novel Approach for Enhancing Reasoning in Large Language Models
by: Tan, Wenting, et al.
Published: (2024)

Embracing Trustworthy Brain-Agent Collaboration as Paradigm Extension for Intelligent Assistive Technologies
by: Chen, Yankai, et al.
Published: (2025)

LLM-Oriented Token-Adaptive Knowledge Distillation
by: Xie, Xurong, et al.
Published: (2025)

Condor: Enhance LLM Alignment with Knowledge-Driven Data Synthesis and Refinement
by: Cao, Maosong, et al.
Published: (2025)

EasyDistill: A Comprehensive Toolkit for Effective Knowledge Distillation of Large Language Models
by: Wang, Chengyu, et al.
Published: (2025)

QUILL: Quotation Generation Enhancement of Large Language Models
by: Xiao, Jin, et al.
Published: (2024)

Cross-Modal Knowledge Distillation for Speech Large Language Models
by: Wang, Enzhi, et al.
Published: (2025)

Distilling Event Sequence Knowledge From Large Language Models
by: Wadhwa, Somin, et al.
Published: (2024)

Mixed Distillation Helps Smaller Language Model Better Reasoning
by: Li, Chenglin, et al.
Published: (2023)

Sens-Merging: Sensitivity-Guided Parameter Balancing for Merging Large Language Models
by: Liu, Shuqi, et al.
Published: (2025)

A Survey on Diffusion Language Models
by: Li, Tianyi, et al.
Published: (2025)

Structured Agent Distillation for Large Language Model
by: Liu, Jun, et al.
Published: (2025)

Large Language Models Meet Knowledge Graphs for Question Answering: Synthesis and Opportunities
by: Ma, Chuangtao, et al.
Published: (2025)

Non-myopic Generation of Language Models for Reasoning and Planning
by: Ma, Chang, et al.
Published: (2024)

Distilling Mathematical Reasoning Capabilities into Small Language Models
by: Zhu, Xunyu, et al.
Published: (2024)

Collaborative Performance Prediction for Large Language Models
by: Zhang, Qiyuan, et al.
Published: (2024)

Towards Understanding and Improving Knowledge Distillation for Neural Machine Translation
by: Zhang, Songming, et al.
Published: (2023)

Learning Evolving Tools for Large Language Models
by: Chen, Guoxin, et al.
Published: (2024)

AdaSVD: Adaptive Singular Value Decomposition for Large Language Models
by: Li, Zhiteng, et al.
Published: (2025)

Advancing Mathematical Reasoning in Language Models: The Impact of Problem-Solving Data, Data Synthesis Methods, and Training Stages
by: Chen, Zui, et al.
Published: (2025)

ELAD: Explanation-Guided Large Language Models Active Distillation
by: Zhang, Yifei, et al.
Published: (2024)

A Survey on Large Language Model based Autonomous Agents
by: Wang, Lei, et al.
Published: (2023)

LLMR: Knowledge Distillation with a Large Language Model-Induced Reward
by: Li, Dongheng, et al.
Published: (2024)

Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Models
by: Wu, Taiqiang, et al.
Published: (2024)

Data Distribution Bottlenecks in Grounding Language Models to Knowledge Bases
by: Shu, Yiheng, et al.
Published: (2023)

OPSDL: On-Policy Self-Distillation for Long-Context Language Models
by: Zhang, Xinsen, et al.
Published: (2026)