Cover Image

A Dual-Space Framework for General Knowledge Distillation of Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Xue, Zhang, Songming, Liang, Yunlong, Meng, Fandong, Chen, Yufeng, Xu, Jinan, Zhou, Jie
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Dual-Space Knowledge Distillation for Large Language Models
by: Zhang, Songming, et al.
Published: (2024)

KDFlow: A User-Friendly and Efficient Knowledge Distillation Framework for Large Language Models
by: Zhang, Songming, et al.
Published: (2026)

Towards Understanding and Improving Knowledge Distillation for Neural Machine Translation
by: Zhang, Songming, et al.
Published: (2023)

CM-Align: Consistency-based Multilingual Alignment for Large Language Models
by: Zhang, Xue, et al.
Published: (2025)

Multilingual Knowledge Editing with Language-Agnostic Factual Neurons
by: Zhang, Xue, et al.
Published: (2024)

AlignDistil: Token-Level Language Model Alignment as Adaptive Policy Distillation
by: Zhang, Songming, et al.
Published: (2025)

Less, but Better: Efficient Multilingual Expansion for LLMs via Layer-wise Mixture-of-Experts
by: Zhang, Xue, et al.
Published: (2025)

LCS: A Language Converter Strategy for Zero-Shot Neural Machine Translation
by: Sun, Zengkui, et al.
Published: (2024)

Outdated Issue Aware Decoding for Reasoning Questions on Edited Knowledge
by: Sun, Zengkui, et al.
Published: (2024)

Cross-Lingual Knowledge Editing in Large Language Models
by: Wang, Jiaan, et al.
Published: (2023)

Think Natively: Unlocking Multilingual Reasoning with Consistency-Enhanced Reinforcement Learning
by: Zhang, Xue, et al.
Published: (2025)

An Empirical Study of Many-to-Many Summarization with Large Language Models
by: Wang, Jiaan, et al.
Published: (2025)

DRT: Deep Reasoning Translation via Long Chain-of-Thought
by: Wang, Jiaan, et al.
Published: (2024)

TEAL: Tokenize and Embed ALL for Multi-modal Large Language Models
by: Yang, Zhen, et al.
Published: (2023)

Warmup-Distill: Bridge the Distribution Mismatch between Teacher and Student before Knowledge Distillation
by: Sun, Zengkui, et al.
Published: (2025)

Enhancing Cross-Tokenizer Knowledge Distillation with Contextual Dynamical Mapping
by: Chen, Yijie, et al.
Published: (2025)

Beyond Next Token Prediction: Patch-Level Training for Large Language Models
by: Shao, Chenze, et al.
Published: (2024)

Retrieval-Augmented Machine Translation with Unstructured Knowledge
by: Wang, Jiaan, et al.
Published: (2024)

Unsupervised Information Refinement Training of Large Language Models for Retrieval-Augmented Generation
by: Xu, Shicheng, et al.
Published: (2024)

TasTe: Teaching Large Language Models to Translate through Self-Reflection
by: Wang, Yutong, et al.
Published: (2024)

EAG: Extract and Generate Multi-way Aligned Corpus for Complete Multi-lingual Neural Machine Translation
by: Xu, Yulin, et al.
Published: (2022)

Continuous Autoregressive Language Models
by: Shao, Chenze, et al.
Published: (2025)

General learned delegation by clones
by: Li, Darren, et al.
Published: (2026)

CSCD-NS: a Chinese Spelling Check Dataset for Native Speakers
by: Hu, Yong, et al.
Published: (2022)

DeepTrans: Deep Reasoning Translation via Reinforcement Learning
by: Wang, Jiaan, et al.
Published: (2025)

ExTrans: Multilingual Deep Reasoning Translation via Exemplar-Enhanced Reinforcement Learning
by: Wang, Jiaan, et al.
Published: (2025)

Pedagogically-Inspired Data Synthesis for Language Model Knowledge Distillation
by: He, Bowei, et al.
Published: (2026)

Knowledge-Driven Agentic Scientific Corpus Distillation Framework for Biomedical Large Language Models Training
by: Xiao, Meng, et al.
Published: (2025)

On Prompt-Driven Safeguarding for Large Language Models
by: Zheng, Chujie, et al.
Published: (2024)

Comments as Natural Logic Pivots: Improve Code Generation via Comment Perspective
by: Chen, Yijie, et al.
Published: (2024)

Translatotron-V(ison): An End-to-End Model for In-Image Machine Translation
by: Lan, Zhibin, et al.
Published: (2024)

ELAD: Explanation-Guided Large Language Models Active Distillation
by: Zhang, Yifei, et al.
Published: (2024)

LLaVE: Large Language and Vision Embedding Models with Hardness-Weighted Contrastive Learning
by: Lan, Zhibin, et al.
Published: (2025)

LLM-KT: Aligning Large Language Models with Knowledge Tracing using a Plug-and-Play Instruction
by: Wang, Ziwei, et al.
Published: (2025)

Less is More: Selective Reflection for Compatible and Efficient Knowledge Distillation in Large Language Models
by: Liu, Lingyuan, et al.
Published: (2025)

Multilingual Collaborative Defense for Large Language Models
by: Li, Hongliang, et al.
Published: (2025)

Survey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Application
by: Yang, Chuanpeng, et al.
Published: (2024)

WTU-EVAL: A Whether-or-Not Tool Usage Evaluation Benchmark for Large Language Models
by: Ning, Kangyun, et al.
Published: (2024)

Contextualization Distillation from Large Language Model for Knowledge Graph Completion
by: Li, Dawei, et al.
Published: (2024)

Pre-training Distillation for Large Language Models: A Design Space Exploration
by: Peng, Hao, et al.
Published: (2024)