Imagen de Portada

Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Fu, Yuqian, Huang, Haohuan, Jiang, Kaiwen, Liu, Jiacai, Jiang, Zhuo, Zhu, Yuanheng, Zhao, Dongbin
Formato:	Preprint
Publicado:	2026
Materias:	Machine Learning Artificial Intelligence Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

DipLLM: Fine-Tuning LLM for Strategic Decision-making in Diplomacy
por: Xu, Kaixuan, et al.
Publicado: (2025)

SRFT: A Single-Stage Method with Supervised and Reinforcement Fine-Tuning for Reasoning
por: Fu, Yuqian, et al.
Publicado: (2025)

Discretizing Continuous Action Space with Unimodal Probability Distributions for On-Policy Reinforcement Learning
por: Zhu, Yuanyang, et al.
Publicado: (2024)

RLAE: Reinforcement Learning-Assisted Ensemble for LLMs
por: Fu, Yuqian, et al.
Publicado: (2025)

Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive
por: Pal, Arka, et al.
Publicado: (2024)

From Signal Degradation to Computation Collapse: Uncovering the Two Failure Modes of LLM Quantization
por: Zhou, Chenxi, et al.
Publicado: (2026)

A Note on Hybrid Online Reinforcement and Imitation Learning for LLMs: Formulations and Algorithms
por: Li, Yingru, et al.
Publicado: (2025)

Beyond Precision: Training-Inference Mismatch is an Optimization Problem and Simple LR Scheduling Fixes It
por: Zhang, Yaxiang, et al.
Publicado: (2026)

GEAR: Granularity-Adaptive Advantage Reweighting for LLM Agents via Self-Distillation
por: Li, Sijia, et al.
Publicado: (2026)

Are Full Rollouts Necessary for On-Policy Distillation?
por: Zhang, Yaocheng, et al.
Publicado: (2026)

Policy Split: Incentivizing Dual-Mode Exploration in LLM Reinforcement with Dual-Mode Entropy Regularization
por: Yao, Jiashu, et al.
Publicado: (2026)

SCOPE: Signal-Calibrated On-Policy Distillation Enhancement with Dual-Path Adaptive Weighting
por: Zheng, Binbin, et al.
Publicado: (2026)

FLEKE: Federated Locate-then-Edit Knowledge Editing
por: Zhao, Zongkai, et al.
Publicado: (2025)

KVSculpt: KV Cache Compression as Distillation
por: Jiang, Bo, et al.
Publicado: (2026)

FM3Q: Factorized Multi-Agent MiniMax Q-Learning for Two-Team Zero-Sum Markov Game
por: Hu, Guangzheng, et al.
Publicado: (2024)

DP-OPD: Differentially Private On-Policy Distillation for Language Models
por: Khadem, Fatemeh, et al.
Publicado: (2026)

Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillation
por: Yang, Zhuolin, et al.
Publicado: (2026)

Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolation
por: Yang, Wenkai, et al.
Publicado: (2026)

AlignDistil: Token-Level Language Model Alignment as Adaptive Policy Distillation
por: Zhang, Songming, et al.
Publicado: (2025)

ORPO-Distill: Mixed-Policy Preference Optimization for Cross-Architecture LLM Distillation
por: Singh, Aasheesh, et al.
Publicado: (2025)

Adaptive Social Learning via Mode Policy Optimization for Language Agents
por: Wang, Minzheng, et al.
Publicado: (2025)

Delta Knowledge Distillation for Large Language Models
por: Cao, Yihan, et al.
Publicado: (2025)

Simple Policy Gradients for Reasoning with Diffusion Language Models
por: Zhan, Anthony
Publicado: (2025)

Revisiting the Capacity Gap in Chain-of-Thought Distillation from a Practical Perspective
por: Kajitsuka, Tokio, et al.
Publicado: (2026)

Cornerstones or Stumbling Blocks? Deciphering the Rock Tokens in On-Policy Distillation
por: Jiang, Yuxuan, et al.
Publicado: (2026)

Bridging Reasoning Trajectories in On-Policy Distillation via Near-Future Guidance
por: Jiang, Yuxuan, et al.
Publicado: (2026)

Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
por: Li, Yaxuan, et al.
Publicado: (2026)

Generalizing Consistency Policy to Visual RL with Prioritized Proximal Experience Regularization
por: Li, Haoran, et al.
Publicado: (2024)

Hybrid Policy Distillation for LLMs
por: Zhu, Wenhong, et al.
Publicado: (2026)

When Stability Fails: Hidden Failure Modes Of LLMS in Data-Constrained Scientific Decision-Making
por: Riasat, Nazia
Publicado: (2026)

CriticSearch: Fine-Grained Credit Assignment for Search Agents via a Retrospective Critic
por: Zhang, Yaocheng, et al.
Publicado: (2025)

Library Drift: Diagnosing and Fixing a Silent Failure Mode in Self-Evolving LLM Skill Libraries
por: Zhang, Xing, et al.
Publicado: (2026)

MedHallBench: A New Benchmark for Assessing Hallucination in Medical Large Language Models
por: Zuo, Kaiwen, et al.
Publicado: (2024)

MAD-OPD: Breaking the Ceiling in On-Policy Distillation via Multi-Agent Debate
por: Wang, Jianze, et al.
Publicado: (2026)

Learning from Failures in Multi-Attempt Reinforcement Learning
por: Chung, Stephen, et al.
Publicado: (2025)

KL for a KL: On-Policy Distillation with Control Variate Baseline
por: Oh, Minjae, et al.
Publicado: (2026)

Aligning Tree-Search Policies with Fixed Token Budgets in Test-Time Scaling of LLMs
por: Miyamoto, Sora, et al.
Publicado: (2026)

Hindsight-Anchored Policy Optimization: Turning Failure into Feedback in Sparse Reward Settings
por: Wu, Yuning, et al.
Publicado: (2026)

Merge-of-Thought Distillation
por: Shen, Zhanming, et al.
Publicado: (2025)

EBPO: Empirical Bayes Shrinkage for Stabilizing Group-Relative Policy Optimization
por: Han, Kevin, et al.
Publicado: (2026)