Copertina

Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Fu, Yuqian, Huang, Haohuan, Jiang, Kaiwen, Liu, Jiacai, Jiang, Zhuo, Zhu, Yuanheng, Zhao, Dongbin
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning Artificial Intelligence Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

DipLLM: Fine-Tuning LLM for Strategic Decision-making in Diplomacy
di: Xu, Kaixuan, et al.
Pubblicazione: (2025)

SRFT: A Single-Stage Method with Supervised and Reinforcement Fine-Tuning for Reasoning
di: Fu, Yuqian, et al.
Pubblicazione: (2025)

Discretizing Continuous Action Space with Unimodal Probability Distributions for On-Policy Reinforcement Learning
di: Zhu, Yuanyang, et al.
Pubblicazione: (2024)

RLAE: Reinforcement Learning-Assisted Ensemble for LLMs
di: Fu, Yuqian, et al.
Pubblicazione: (2025)

Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive
di: Pal, Arka, et al.
Pubblicazione: (2024)

From Signal Degradation to Computation Collapse: Uncovering the Two Failure Modes of LLM Quantization
di: Zhou, Chenxi, et al.
Pubblicazione: (2026)

A Note on Hybrid Online Reinforcement and Imitation Learning for LLMs: Formulations and Algorithms
di: Li, Yingru, et al.
Pubblicazione: (2025)

Beyond Precision: Training-Inference Mismatch is an Optimization Problem and Simple LR Scheduling Fixes It
di: Zhang, Yaxiang, et al.
Pubblicazione: (2026)

GEAR: Granularity-Adaptive Advantage Reweighting for LLM Agents via Self-Distillation
di: Li, Sijia, et al.
Pubblicazione: (2026)

Are Full Rollouts Necessary for On-Policy Distillation?
di: Zhang, Yaocheng, et al.
Pubblicazione: (2026)

Policy Split: Incentivizing Dual-Mode Exploration in LLM Reinforcement with Dual-Mode Entropy Regularization
di: Yao, Jiashu, et al.
Pubblicazione: (2026)

SCOPE: Signal-Calibrated On-Policy Distillation Enhancement with Dual-Path Adaptive Weighting
di: Zheng, Binbin, et al.
Pubblicazione: (2026)

FLEKE: Federated Locate-then-Edit Knowledge Editing
di: Zhao, Zongkai, et al.
Pubblicazione: (2025)

KVSculpt: KV Cache Compression as Distillation
di: Jiang, Bo, et al.
Pubblicazione: (2026)

FM3Q: Factorized Multi-Agent MiniMax Q-Learning for Two-Team Zero-Sum Markov Game
di: Hu, Guangzheng, et al.
Pubblicazione: (2024)

DP-OPD: Differentially Private On-Policy Distillation for Language Models
di: Khadem, Fatemeh, et al.
Pubblicazione: (2026)

Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillation
di: Yang, Zhuolin, et al.
Pubblicazione: (2026)

Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolation
di: Yang, Wenkai, et al.
Pubblicazione: (2026)

AlignDistil: Token-Level Language Model Alignment as Adaptive Policy Distillation
di: Zhang, Songming, et al.
Pubblicazione: (2025)

ORPO-Distill: Mixed-Policy Preference Optimization for Cross-Architecture LLM Distillation
di: Singh, Aasheesh, et al.
Pubblicazione: (2025)

Adaptive Social Learning via Mode Policy Optimization for Language Agents
di: Wang, Minzheng, et al.
Pubblicazione: (2025)

Delta Knowledge Distillation for Large Language Models
di: Cao, Yihan, et al.
Pubblicazione: (2025)

Simple Policy Gradients for Reasoning with Diffusion Language Models
di: Zhan, Anthony
Pubblicazione: (2025)

Revisiting the Capacity Gap in Chain-of-Thought Distillation from a Practical Perspective
di: Kajitsuka, Tokio, et al.
Pubblicazione: (2026)

Cornerstones or Stumbling Blocks? Deciphering the Rock Tokens in On-Policy Distillation
di: Jiang, Yuxuan, et al.
Pubblicazione: (2026)

Bridging Reasoning Trajectories in On-Policy Distillation via Near-Future Guidance
di: Jiang, Yuxuan, et al.
Pubblicazione: (2026)

Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
di: Li, Yaxuan, et al.
Pubblicazione: (2026)

Generalizing Consistency Policy to Visual RL with Prioritized Proximal Experience Regularization
di: Li, Haoran, et al.
Pubblicazione: (2024)

Hybrid Policy Distillation for LLMs
di: Zhu, Wenhong, et al.
Pubblicazione: (2026)

When Stability Fails: Hidden Failure Modes Of LLMS in Data-Constrained Scientific Decision-Making
di: Riasat, Nazia
Pubblicazione: (2026)

CriticSearch: Fine-Grained Credit Assignment for Search Agents via a Retrospective Critic
di: Zhang, Yaocheng, et al.
Pubblicazione: (2025)

Library Drift: Diagnosing and Fixing a Silent Failure Mode in Self-Evolving LLM Skill Libraries
di: Zhang, Xing, et al.
Pubblicazione: (2026)

MAD-OPD: Breaking the Ceiling in On-Policy Distillation via Multi-Agent Debate
di: Wang, Jianze, et al.
Pubblicazione: (2026)

MedHallBench: A New Benchmark for Assessing Hallucination in Medical Large Language Models
di: Zuo, Kaiwen, et al.
Pubblicazione: (2024)

Learning from Failures in Multi-Attempt Reinforcement Learning
di: Chung, Stephen, et al.
Pubblicazione: (2025)

KL for a KL: On-Policy Distillation with Control Variate Baseline
di: Oh, Minjae, et al.
Pubblicazione: (2026)

Aligning Tree-Search Policies with Fixed Token Budgets in Test-Time Scaling of LLMs
di: Miyamoto, Sora, et al.
Pubblicazione: (2026)

Hindsight-Anchored Policy Optimization: Turning Failure into Feedback in Sparse Reward Settings
di: Wu, Yuning, et al.
Pubblicazione: (2026)

Merge-of-Thought Distillation
di: Shen, Zhanming, et al.
Pubblicazione: (2025)

EBPO: Empirical Bayes Shrinkage for Stabilizing Group-Relative Policy Optimization
di: Han, Kevin, et al.
Pubblicazione: (2026)