Copertina

KDFlow: A User-Friendly and Efficient Knowledge Distillation Framework for Large Language Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Zhang, Songming, Zhang, Xue, Zhang, Tong, Hu, Bojie, Chen, Yufeng, Xu, Jinan
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Computation and Language Artificial Intelligence Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

AlignDistil: Token-Level Language Model Alignment as Adaptive Policy Distillation
di: Zhang, Songming, et al.
Pubblicazione: (2025)

A Dual-Space Framework for General Knowledge Distillation of Large Language Models
di: Zhang, Xue, et al.
Pubblicazione: (2025)

Dual-Space Knowledge Distillation for Large Language Models
di: Zhang, Songming, et al.
Pubblicazione: (2024)

Towards Understanding and Improving Knowledge Distillation for Neural Machine Translation
di: Zhang, Songming, et al.
Pubblicazione: (2023)

LLM-NEO: Parameter Efficient Knowledge Distillation for Large Language Models
di: Yang, Runming, et al.
Pubblicazione: (2024)

Being Strong Progressively! Enhancing Knowledge Distillation of Large Language Models through a Curriculum Learning Framework
di: Liu, Lingyuan, et al.
Pubblicazione: (2025)

PromptKD: Distilling Student-Friendly Knowledge for Generative Language Models via Prompt Tuning
di: Kim, Gyeongman, et al.
Pubblicazione: (2024)

Delta Knowledge Distillation for Large Language Models
di: Cao, Yihan, et al.
Pubblicazione: (2025)

Efficient Mathematical Reasoning Models via Dynamic Pruning and Knowledge Distillation
di: Yu, Fengming, et al.
Pubblicazione: (2025)

Structured Agent Distillation for Large Language Model
di: Liu, Jun, et al.
Pubblicazione: (2025)

KGLens: Towards Efficient and Effective Knowledge Probing of Large Language Models with Knowledge Graphs
di: Zheng, Shangshang, et al.
Pubblicazione: (2023)

An Efficient Inference Framework for Early-exit Large Language Models
di: Miao, Ruijie, et al.
Pubblicazione: (2024)

KIEval: A Knowledge-grounded Interactive Evaluation Framework for Large Language Models
di: Yu, Zhuohao, et al.
Pubblicazione: (2024)

Large Language Models Explore by Latent Distilling
di: Zeng, Yuanhao, et al.
Pubblicazione: (2026)

Evaluating the Factuality of Large Language Models using Large-Scale Knowledge Graphs
di: Liu, Xiaoze, et al.
Pubblicazione: (2024)

Sliding Window Attention Training for Efficient Large Language Models
di: Fu, Zichuan, et al.
Pubblicazione: (2025)

LLM Surgery: Efficient Knowledge Unlearning and Editing in Large Language Models
di: Veldanda, Akshaj Kumar, et al.
Pubblicazione: (2024)

Towards the Law of Capacity Gap in Distilling Language Models
di: Zhang, Chen, et al.
Pubblicazione: (2023)

Collaborative Performance Prediction for Large Language Models
di: Zhang, Qiyuan, et al.
Pubblicazione: (2024)

EfficientQAT: Efficient Quantization-Aware Training for Large Language Models
di: Chen, Mengzhao, et al.
Pubblicazione: (2024)

TAID: Temporally Adaptive Interpolated Distillation for Efficient Knowledge Transfer in Language Models
di: Shing, Makoto, et al.
Pubblicazione: (2025)

SDMPrune: Self-Distillation MLP Pruning for Efficient Large Language Models
di: Zhu, Hourun, et al.
Pubblicazione: (2025)

AdaSPEC: Selective Knowledge Distillation for Efficient Speculative Decoders
di: Hu, Yuezhou, et al.
Pubblicazione: (2025)

FactorLLM: Factorizing Knowledge via Mixture of Experts for Large Language Models
di: Zhao, Zhongyu, et al.
Pubblicazione: (2024)

An Experimental Design Framework for Label-Efficient Supervised Finetuning of Large Language Models
di: Bhatt, Gantavya, et al.
Pubblicazione: (2024)

ME-Switch: A Memory-Efficient Expert Switching Framework for Large Language Models
di: Liu, Jing, et al.
Pubblicazione: (2024)

Compact Language Models via Pruning and Knowledge Distillation
di: Muralidharan, Saurav, et al.
Pubblicazione: (2024)

$\mathbf{(N,K)}$-Puzzle: A Cost-Efficient Testbed for Benchmarking Reinforcement Learning Algorithms in Generative Language Model
di: Zhang, Yufeng, et al.
Pubblicazione: (2024)

Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
di: Li, Yaxuan, et al.
Pubblicazione: (2026)

Beyond Mimicry to Contextual Guidance: Knowledge Distillation for Interactive AI
di: Wang, Tong, et al.
Pubblicazione: (2024)

BESA: Pruning Large Language Models with Blockwise Parameter-Efficient Sparsity Allocation
di: Xu, Peng, et al.
Pubblicazione: (2024)

UniSD: Towards a Unified Self-Distillation Framework for Large Language Models
di: Jin, Yiqiao, et al.
Pubblicazione: (2026)

Incompressible Knowledge Probes: Estimating Black-Box LLM Parameter Counts via Factual Capacity
di: Li, Bojie
Pubblicazione: (2026)

DistiLLM: Towards Streamlined Distillation for Large Language Models
di: Ko, Jongwoo, et al.
Pubblicazione: (2024)

HMI: Hierarchical Knowledge Management for Efficient Multi-Tenant Inference in Pretrained Language Models
di: Zhang, Jun, et al.
Pubblicazione: (2025)

Precise Attribute Intensity Control in Large Language Models via Targeted Representation Editing
di: Zhang, Rongzhi, et al.
Pubblicazione: (2025)

Federated Learning with Layer Skipping: Efficient Training of Large Language Models for Healthcare NLP
di: Zhang, Lihong, et al.
Pubblicazione: (2025)

Knowledgeable Agents by Offline Reinforcement Learning from Large Language Model Rollouts
di: Pang, Jing-Cheng, et al.
Pubblicazione: (2024)

Dist2ill: Distributional Distillation for One-Pass Uncertainty Estimation in Large Language Models
di: Zhao, Yicong, et al.
Pubblicazione: (2025)

LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning
di: Pan, Rui, et al.
Pubblicazione: (2024)