Cover Image

From Generic Correlation to Input-Specific Credit in On-Policy Self Distillation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Shen, Guobin, Huang, Lei, Cheng, Xiang, Zhao, Chenxiao, Li, Jindong, Zhao, Dongcheng, Yu, Xing
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Anti-Self-Distillation for Reasoning RL via Pointwise Mutual Information
by: Shen, Guobin, et al.
Published: (2026)

VESPO: Variational Sequence-Level Soft Policy Optimization for Stable Off-Policy LLM Training
by: Shen, Guobin, et al.
Published: (2026)

Multi-Level Safety Continual Projection for Fine-Tuned Large Language Models without Retraining
by: Han, Bing, et al.
Published: (2025)

Light Alignment Improves LLM Safety via Model Self-Reflection with a Single Neuron
by: Shen, Sicheng, et al.
Published: (2026)

Safety Instincts: LLMs Learn to Trust Their Internal Compass for Self-Defense
by: Shen, Guobin, et al.
Published: (2025)

OISD: On-Policy Internal Self-Distillation of Language Models
by: Liu, Xinyu, et al.
Published: (2026)

Self-ReSET: Learning to Self-Recover from Unsafe Reasoning Trajectories
by: Zhang, Dongcheng, et al.
Published: (2026)

PACED: Distillation and On-Policy Self-Distillation at the Frontier of Student Competence
by: Xu, Yuanda, et al.
Published: (2026)

Online Policy Distillation with Decision-Attention
by: Yu, Xinqiang, et al.
Published: (2024)

HDPO: Hybrid Distillation Policy Optimization via Privileged Self-Distillation
by: Ding, Ken
Published: (2026)

A General Framework for Learning from Weak Supervision
by: Chen, Hao, et al.
Published: (2024)

OPD+: Rethinking the Advantage Design for On-Policy Distillation
by: Zhao, Hanyang, et al.
Published: (2026)

Towards Reliable Evaluation of Adversarial Robustness for Spiking Neural Networks
by: Wang, Jihang, et al.
Published: (2025)

On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes
by: Agarwal, Rishabh, et al.
Published: (2023)

Beyond Uniform Credit: Causal Credit Assignment for Policy Optimization
by: Khandoga, Mykola, et al.
Published: (2026)

Implicit Federated In-context Learning For Task-Specific LLM Fine-Tuning
by: Li, Dongcheng, et al.
Published: (2025)

Graph Dimension Attention Networks for Enterprise Credit Assessment
by: Wei, Shaopeng, et al.
Published: (2024)

Multimodal Generative Engine Optimization: Rank Manipulation for Vision-Language Model Rankers
by: Du, Yixuan, et al.
Published: (2026)

Self-Supervised Quantization-Aware Knowledge Distillation
by: Zhao, Kaiqi, et al.
Published: (2024)

FedSDR: Federated Self-Distillation with Rectification
by: Ren, Ziheng, et al.
Published: (2026)

Bootstrapping Exploration with Group-Level Natural Language Feedback in Reinforcement Learning
by: Huang, Lei, et al.
Published: (2026)

Proximal Policy Distillation
by: Spigler, Giacomo
Published: (2024)

When Are Teacher Tokens Reliable? Position-Weighted On-Policy Self-Distillation for Reasoning
by: Liu, Xiaogeng, et al.
Published: (2026)

Unifying Group-Relative and Self-Distillation Policy Optimization via Sample Routing
by: Li, Gengsheng, et al.
Published: (2026)

TRACE: Distilling Where It Matters via Token-Routed Self On-Policy Alignment
by: Wang, Jiaxuan, et al.
Published: (2026)

Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes
by: Fu, Yuqian, et al.
Published: (2026)

Asymmetric On-Policy Distillation: Bridging Exploitation and Imitation at the Token Level
by: Jia, Nan, et al.
Published: (2026)

PromptBench: A Unified Library for Evaluation of Large Language Models
by: Zhu, Kaijie, et al.
Published: (2023)

Dynamic Evaluation of Large Language Models by Meta Probing Agents
by: Zhu, Kaijie, et al.
Published: (2024)

Prompt Tuning with Diffusion for Few-Shot Pre-trained Policy Generalization
by: Hu, Shengchao, et al.
Published: (2024)

DGPO: Distribution Guided Policy Optimization for Fine Grained Credit Assignment
by: Jin, Hongbo, et al.
Published: (2026)

Generalized Policy Gradient with History-Aware Decision Transformer for Reliable Routing over Graph Signals
by: Wei, Xing, et al.
Published: (2025)

BPL: Bias-adaptive Preference Distillation Learning for Recommender System
by: Kang, SeongKu, et al.
Published: (2025)

OGLS-SD: On-Policy Self-Distillation with Outcome-Guided Logit Steering for LLM Reasoning
by: Yang, Yuxiao, et al.
Published: (2026)

Provable and Practical In-Context Policy Optimization for Self-Improvement
by: Yu, Tianrun, et al.
Published: (2026)

Annealing Self-Distillation Rectification Improves Adversarial Training
by: Wu, Yu-Yu, et al.
Published: (2023)

Model-Based Proactive Cost Generation for Learning Safe Policies Offline with Limited Violation Data
by: Xue, Ruiqi, et al.
Published: (2026)

Extreme Region Policy Distillation
by: Chen, Changyu, et al.
Published: (2026)

Towards Better Generalization via Distributional Input Projection Network
by: Hao, Yifan, et al.
Published: (2025)

Self-Distillation for Multi-Token Prediction
by: Zhao, Guoliang, et al.
Published: (2026)