Imagen de Portada

A Survey of On-Policy Distillation for Large Language Models

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Song, Mingyang, Zheng, Mao
Formato:	Preprint
Publicado:	2026
Materias:	Machine Learning Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

A Survey on Symbolic Knowledge Distillation of Large Language Models
por: Acharya, Kamal, et al.
Publicado: (2024)

A Survey of Query Optimization in Large Language Models
por: Song, Mingyang, et al.
Publicado: (2024)

Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models
por: Zhao, Siyan, et al.
Publicado: (2026)

SODA: Semi On-Policy Black-Box Distillation for Large Language Models
por: Chen, Xiwen, et al.
Publicado: (2026)

Distilling Large Language Models for Text-Attributed Graph Learning
por: Pan, Bo, et al.
Publicado: (2024)

Towards Lifelong Learning of Large Language Models: A Survey
por: Zheng, Junhao, et al.
Publicado: (2024)

Feature Alignment and Representation Transfer in Knowledge Distillation for Large Language Models
por: Yang, Junjie, et al.
Publicado: (2025)

Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
por: Li, Yaxuan, et al.
Publicado: (2026)

Large Language Models Explore by Latent Distilling
por: Zeng, Yuanhao, et al.
Publicado: (2026)

A Survey on LoRA of Large Language Models
por: Mao, Yuren, et al.
Publicado: (2024)

Model Merging in the Era of Large Language Models: Methods, Applications, and Future Directions
por: Song, Mingyang, et al.
Publicado: (2026)

Adversarial Moment-Matching Distillation of Large Language Models
por: Jia, Chen
Publicado: (2024)

ROSD: Reflective On-Policy Self-Distillation for Language Model Reasoning across Domains
por: Zhao, Ziqi, et al.
Publicado: (2026)

SSR-Zero: Simple Self-Rewarding Reinforcement Learning for Machine Translation
por: Yang, Wenjie, et al.
Publicado: (2025)

Knowledge Distillation from Large Language Models for Household Energy Modeling
por: Takrouri, Mohannad, et al.
Publicado: (2025)

AlignDistil: Token-Level Language Model Alignment as Adaptive Policy Distillation
por: Zhang, Songming, et al.
Publicado: (2025)

Knowledge Distillation and Dataset Distillation of Large Language Models: Emerging Trends, Challenges, and Future Directions
por: Fang, Luyang, et al.
Publicado: (2025)

Cache & Distil: Optimising API Calls to Large Language Models
por: Ramírez, Guillem, et al.
Publicado: (2023)

A Survey on Mixture of Experts in Large Language Models
por: Cai, Weilin, et al.
Publicado: (2024)

Large Language Models on Graphs: A Comprehensive Survey
por: Jin, Bowen, et al.
Publicado: (2023)

Continual Learning for Large Language Models: A Survey
por: Wu, Tongtong, et al.
Publicado: (2024)

Advancing Graph Representation Learning with Large Language Models: A Comprehensive Survey of Techniques
por: Mao, Qiheng, et al.
Publicado: (2024)

A Systematic Survey on Large Language Models for Algorithm Design
por: Liu, Fei, et al.
Publicado: (2024)

Self-Data Distillation for Recovering Quality in Pruned Large Language Models
por: Thangarasa, Vithursan, et al.
Publicado: (2024)

Iterative Layer-wise Distillation for Efficient Compression of Large Language Models
por: Kovalev, Grigory, et al.
Publicado: (2025)

A Survey on Training-free Alignment of Large Language Models
por: Pan, Birong, et al.
Publicado: (2025)

DP-OPD: Differentially Private On-Policy Distillation for Language Models
por: Khadem, Fatemeh, et al.
Publicado: (2026)

Delta Knowledge Distillation for Large Language Models
por: Cao, Yihan, et al.
Publicado: (2025)

Structured Agent Distillation for Large Language Model
por: Liu, Jun, et al.
Publicado: (2025)

MiniDisc: Minimal Distillation Schedule for Language Model Compression
por: Zhang, Chen, et al.
Publicado: (2022)

UniDetox: Universal Detoxification of Large Language Models via Dataset Distillation
por: Lu, Huimin, et al.
Publicado: (2025)

Bypass Back-propagation: Optimization-based Structural Pruning for Large Language Models via Policy Gradient
por: Gao, Yuan, et al.
Publicado: (2024)

Data Mixing for Large Language Models Pretraining: A Survey and Outlook
por: Chen, Zhuo, et al.
Publicado: (2026)

Trust Region On-Policy Distillation
por: Xing, Xingrun, et al.
Publicado: (2026)

Recurrent Confidence Chain: Temporal-Aware Uncertainty Quantification in Large Language Models
por: Mao, Zhenjiang, et al.
Publicado: (2026)

On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes
por: Agarwal, Rishabh, et al.
Publicado: (2023)

Harnessing Large Language Models for Disaster Management: A Survey
por: Lei, Zhenyu, et al.
Publicado: (2025)

Few-Step Diffusion Language Models via Trajectory Self-Distillation
por: Zhang, Tunyu, et al.
Publicado: (2026)

Reasoning-preserved Efficient Distillation of Large Language Models via Activation-aware Initialization
por: He, Junlin, et al.
Publicado: (2026)

Multimodal Large Language Models for Medicine: A Comprehensive Survey
por: Ye, Jiarui, et al.
Publicado: (2025)