Cover Image

MiniLLM: On-Policy Distillation of Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Gu, Yuxian, Dong, Li, Wei, Furu, Huang, Minlie
Format:	Preprint
Published:	2023
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Black-Box On-Policy Distillation of Large Language Models
by: Ye, Tianzhu, et al.
Published: (2025)

MiniPLM: Knowledge Distillation for Pre-Training Language Models
by: Gu, Yuxian, et al.
Published: (2024)

Towards Optimal Learning of Language Models
by: Gu, Yuxian, et al.
Published: (2024)

Direct Preference Knowledge Distillation for Large Language Models
by: Li, Yixing, et al.
Published: (2024)

Data Selection via Optimal Control for Language Models
by: Gu, Yuxian, et al.
Published: (2024)

Self-Boosting Large Language Models with Synthetic Preference Data
by: Dong, Qingxiu, et al.
Published: (2024)

Instruction Pre-Training: Language Models are Supervised Multitask Learners
by: Cheng, Daixuan, et al.
Published: (2024)

Textual Aesthetics in Large Language Models
by: Jiang, Lingjie, et al.
Published: (2024)

LogicGame: Benchmarking Rule-Based Reasoning Abilities of Large Language Models
by: Gui, Jiayi, et al.
Published: (2024)

Computer Environments Elicit General Agentic Intelligence in LLMs
by: Cheng, Daixuan, et al.
Published: (2026)

LongSafety: Evaluating Long-Context Safety of Large Language Models
by: Lu, Yida, et al.
Published: (2025)

The Era of Agentic Organization: Learning to Organize with Language Models
by: Chi, Zewen, et al.
Published: (2025)

CritiqueLLM: Towards an Informative Critique Generation Model for Evaluation of Large Language Model Generation
by: Ke, Pei, et al.
Published: (2023)

Understanding the Dilemma of Unlearning for Large Language Models
by: Zhang, Qingjie, et al.
Published: (2025)

OPSDL: On-Policy Self-Distillation for Long-Context Language Models
by: Zhang, Xinsen, et al.
Published: (2026)

Structured Agent Distillation for Large Language Model
by: Liu, Jun, et al.
Published: (2025)

On-Policy Context Distillation for Language Models
by: Ye, Tianzhu, et al.
Published: (2026)

Perception of Knowledge Boundary for Large Language Models through Semi-open-ended Question Answering
by: Wen, Zhihua, et al.
Published: (2024)

LongReasonArena: A Long Reasoning Benchmark for Large Language Models
by: Ding, Jiayu, et al.
Published: (2025)

AutoDetect: Towards a Unified Framework for Automated Weakness Detection in Large Language Models
by: Cheng, Jiale, et al.
Published: (2024)

QCRD: Quality-guided Contrastive Rationale Distillation for Large Language Models
by: Wang, Wei, et al.
Published: (2024)

Capturing Nuanced Preferences: Preference-Aligned Distillation for Small Language Models
by: Gu, Yanggan, et al.
Published: (2025)

On Prompt-Driven Safeguarding for Large Language Models
by: Zheng, Chujie, et al.
Published: (2024)

Language Models as Inductive Reasoners
by: Yang, Zonglin, et al.
Published: (2022)

SPaR: Self-Play with Tree-Search Refinement to Improve Instruction-Following in Large Language Models
by: Cheng, Jiale, et al.
Published: (2024)

EasyDistill: A Comprehensive Toolkit for Effective Knowledge Distillation of Large Language Models
by: Wang, Chengyu, et al.
Published: (2025)

Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
by: Li, Yaxuan, et al.
Published: (2026)

ReAD: Reinforcement-Guided Capability Distillation for Large Language Models
by: Cheng, Xueqi, et al.
Published: (2026)

FedCoT: Federated Chain-of-Thought Distillation for Large Language Models
by: Fan, Tao, et al.
Published: (2024)

EmoBench: Evaluating the Emotional Intelligence of Large Language Models
by: Sabour, Sahand, et al.
Published: (2024)

Internal Value Alignment in Large Language Models through Controlled Value Vector Activation
by: Jin, Haoran, et al.
Published: (2025)

DistiLLM: Towards Streamlined Distillation for Large Language Models
by: Ko, Jongwoo, et al.
Published: (2024)

From Correction to Mastery: Reinforced Distillation of Large Language Model Agents
by: Lyu, Yuanjie, et al.
Published: (2025)

K-Level Reasoning: Establishing Higher Order Beliefs in Large Language Models for Strategic Reasoning
by: Zhang, Yadong, et al.
Published: (2024)

Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration
by: Wang, Zhenhailong, et al.
Published: (2023)

ToMBench: Benchmarking Theory of Mind in Large Language Models
by: Chen, Zhuang, et al.
Published: (2024)

Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning
by: Yang, Wenkai, et al.
Published: (2025)

UNDIAL: Self-Distillation with Adjusted Logits for Robust Unlearning in Large Language Models
by: Dong, Yijiang River, et al.
Published: (2024)

ChemLLM: A Chemical Large Language Model
by: Zhang, Di, et al.
Published: (2024)

LANTERN: Scalable Distillation of Large Language Models for Job-Person Fit and Explanation
by: Fu, Zhoutong, et al.
Published: (2025)