Image de couverture de livre

Hybrid Policy Distillation for LLMs

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zhu, Wenhong, Xie, Ruobing, Wang, Rui, Liu, Pengfei
Format:	Preprint
Publié:	2026
Sujets:	Computation and Language Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Flexible Realignment of Language Models
par: Zhu, Wenhong, et autres
Publié: (2025)

Proximal Supervised Fine-Tuning
par: Zhu, Wenhong, et autres
Publié: (2025)

Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolation
par: Yang, Wenkai, et autres
Publié: (2026)

Self-Distillation for Multi-Token Prediction
par: Zhao, Guoliang, et autres
Publié: (2026)

SafeSteer: Localized On-Policy Distillation for Efficient Safety Alignment
par: Li, Hao, et autres
Publié: (2026)

Beyond Natural Language: LLMs Leveraging Alternative Formats for Enhanced Reasoning and Communication
par: Chen, Weize, et autres
Publié: (2024)

ALTER: Asymmetric LoRA for Token-Entropy-Guided Unlearning of LLMs
par: Chen, Xunlei, et autres
Publié: (2026)

Language Models "Grok" to Copy
par: Lv, Ang, et autres
Publié: (2024)

CycleDistill: Bootstrapping Machine Translation using LLMs with Cyclical Distillation
par: Halder, Deepon, et autres
Publié: (2025)

Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillation
par: Yang, Zhuolin, et autres
Publié: (2026)

Optimizing Psychological Counseling with Instruction-Tuned Large Language Models
par: Li, Wenjie, et autres
Publié: (2024)

Distill-C: Enhanced NL2SQL via Distilled Customization with LLMs
par: Hoang, Cong Duy Vu, et autres
Publié: (2025)

Same Evidence, Different Answers: Canonical-Context On-Policy Distillation for Multi-Turn Language Models
par: Lin, Zizhuo, et autres
Publié: (2026)

EchoDistill:Alignment Noisy-to-Clean Self-Distillation for Robust Audio LLMs
par: Lin, Liang, et autres
Publié: (2026)

MAD-OPD: Breaking the Ceiling in On-Policy Distillation via Multi-Agent Debate
par: Wang, Jianze, et autres
Publié: (2026)

TestAgent: Automatic Benchmarking and Exploratory Interaction for Evaluating LLMs in Vertical Domains
par: Wang, Wanying, et autres
Publié: (2024)

Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes
par: Fu, Yuqian, et autres
Publié: (2026)

When Perplexity Lies: Generation-Focused Distillation of Hybrid Sequence Models
par: Kostelec, Juan Gabriel, et autres
Publié: (2026)

Distilling Closed-Source LLM's Knowledge for Locally Stable and Economic Biomedical Entity Linking
par: Ai, Yihao, et autres
Publié: (2025)

MiniLLM: On-Policy Distillation of Large Language Models
par: Gu, Yuxian, et autres
Publié: (2023)

Black-Box On-Policy Distillation of Large Language Models
par: Ye, Tianzhu, et autres
Publié: (2025)

On Evaluating LLM Alignment by Evaluating LLMs as Judges
par: Liu, Yixin, et autres
Publié: (2025)

Distilling the Essence: Efficient Reasoning Distillation via Sequence Truncation
par: Chen, Wei-Rui, et autres
Publié: (2025)

Distilling Mathematical Reasoning Capabilities into Small Language Models
par: Zhu, Xunyu, et autres
Publié: (2024)

X-OPD: Cross-Modal On-Policy Distillation for Capability Alignment in Speech LLMs
par: Cao, Di, et autres
Publié: (2026)

Multi-Stage Balanced Distillation: Addressing Long-Tail Challenges in Sequence-Level Knowledge Distillation
par: Zhou, Yuhang, et autres
Publié: (2024)

OPSDL: On-Policy Self-Distillation for Long-Context Language Models
par: Zhang, Xinsen, et autres
Publié: (2026)

Cornerstones or Stumbling Blocks? Deciphering the Rock Tokens in On-Policy Distillation
par: Jiang, Yuxuan, et autres
Publié: (2026)

Can Large Language Models be Trusted for Evaluation? Scalable Meta-Evaluation of LLMs as Evaluators via Agent Debate
par: Chern, Steffi, et autres
Publié: (2024)

LLM-Oriented Token-Adaptive Knowledge Distillation
par: Xie, Xurong, et autres
Publié: (2025)

KV-Distill: Nearly Lossless Learnable Context Compression for LLMs
par: Chari, Vivek, et autres
Publié: (2025)

Distilling Text Style Transfer With Self-Explanation From LLMs
par: Zhang, Chiyu, et autres
Publié: (2024)

SelecTKD: Selective Token-Weighted Knowledge Distillation for LLMs
par: Huang, Haiduo, et autres
Publié: (2025)

Probing to Refine: Reinforcement Distillation of LLMs via Explanatory Inversion
par: Tan, Zhen, et autres
Publié: (2026)

Lossless KV Cache Compression to 2%
par: Yang, Zhen, et autres
Publié: (2024)

Pioneering Reliable Assessment in Text-to-Image Knowledge Editing: Leveraging a Fine-Grained Dataset and an Innovative Criterion
par: Gu, Hengrui, et autres
Publié: (2024)

Surgical Post-Training: Proximal On-Policy Distillation for Reasoning with Knowledge Retention
par: Lin, Wenye, et autres
Publié: (2026)

Bridging Reasoning Trajectories in On-Policy Distillation via Near-Future Guidance
par: Jiang, Yuxuan, et autres
Publié: (2026)

Distilling to Hybrid Attention Models via KL-Guided Layer Selection
par: Li, Yanhong, et autres
Publié: (2025)

Your Teacher Can't Help You Here: Combating Supervision Fidelity Decay in On-Policy Distillation
par: Liu, Yanjiang, et autres
Publié: (2026)