Cover Image

TIP: Token Importance in On-Policy Distillation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Xu, Yuanda, Sang, Hejian, Zhou, Zhengze, He, Ran, Wang, Zhipeng, Geramifard, Alborz
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Beyond GRPO and On-Policy Distillation: An Empirical Sparse-to-Dense Reward Principle for Language-Model Post-Training
by: Xu, Yuanda, et al.
Published: (2026)

PACED: Distillation and On-Policy Self-Distillation at the Frontier of Student Competence
by: Xu, Yuanda, et al.
Published: (2026)

Overconfident Errors Need Stronger Correction: Asymmetric Confidence Penalties for Reinforcement Learning
by: Xu, Yuanda, et al.
Published: (2026)

Not all tokens are needed(NAT): token efficient reinforcement learning
by: Sang, Hejian, et al.
Published: (2026)

CRISP: Compressed Reasoning via Iterative Self-Policy Distillation
by: Sang, Hejian, et al.
Published: (2026)

SODA: Semi On-Policy Black-Box Distillation for Large Language Models
by: Chen, Xiwen, et al.
Published: (2026)

Robust Batch-Level Query Routing for Large Language Models under Cost and Capacity Constraints
by: Markovic-Voronov, Jelena, et al.
Published: (2026)

When should we prefer Decision Transformers for Offline Reinforcement Learning?
by: Bhargava, Prajjwal, et al.
Published: (2023)

Debunk the Myth of SFT Generalization
by: Lin, Xiaofeng, et al.
Published: (2025)

Planner-R1: Reward Shaping Enables Efficient Agentic RL with Smaller LLMs
by: Zhu, Siyu, et al.
Published: (2025)

SMORE: Score Models for Offline Goal-Conditioned Reinforcement Learning
by: Sikchi, Harshit, et al.
Published: (2023)

Aligning Diffusion Language Models via Unpaired Preference Optimization
by: Jindal, Vaibhav, et al.
Published: (2025)

Agentic Reinforcement Learning for Real-World Code Repair
by: Zhu, Siyu, et al.
Published: (2025)

Rethinking Importance Sampling in LLM Policy Optimization: A Cumulative Token Perspective
by: Zhang, Yuheng, et al.
Published: (2026)

AlignDistil: Token-Level Language Model Alignment as Adaptive Policy Distillation
by: Zhang, Songming, et al.
Published: (2025)

Asymmetric On-Policy Distillation: Bridging Exploitation and Imitation at the Token Level
by: Jia, Nan, et al.
Published: (2026)

Sequential Decision-Making for Inline Text Autocomplete
by: Chitnis, Rohan, et al.
Published: (2024)

When Are Teacher Tokens Reliable? Position-Weighted On-Policy Self-Distillation for Reasoning
by: Liu, Xiaogeng, et al.
Published: (2026)

TRACE: Distilling Where It Matters via Token-Routed Self On-Policy Alignment
by: Wang, Jiaxuan, et al.
Published: (2026)

Scaling In-Context Online Learning Capability of LLMs via Cross-Episode Meta-RL
by: Lin, Xiaofeng, et al.
Published: (2026)

TokenButler: Token Importance is Predictable
by: Akhauri, Yash, et al.
Published: (2025)

GIPO: Gaussian Importance Sampling Policy Optimization
by: Lu, Chengxuan, et al.
Published: (2026)

Extreme Region Policy Distillation
by: Chen, Changyu, et al.
Published: (2026)

daDPO: Distribution-Aware DPO for Distilling Conversational Abilities
by: Zhang, Zhengze, et al.
Published: (2025)

Post-Training is About States, Not Tokens: A State Distribution View of SFT, RL, and On-Policy Distillation
by: Nie, Dong
Published: (2026)

Demystifying the Paradox of Importance Sampling with an Estimated History-Dependent Behavior Policy in Off-Policy Evaluation
by: Zhou, Hongyi, et al.
Published: (2025)

Online Policy Distillation with Decision-Attention
by: Yu, Xinqiang, et al.
Published: (2024)

Proximal Policy Distillation
by: Spigler, Giacomo
Published: (2024)

ESPO: Entropy Importance Sampling Policy Optimization
by: Sheng, Yuepeng, et al.
Published: (2025)

Relative Importance Sampling for off-Policy Actor-Critic in Deep Reinforcement Learning
by: Humayoo, Mahammad, et al.
Published: (2018)

Quotient DAGs for Off-Policy Evaluation:Forward-Flow Importance Sampling and Exact Slate Propensities
by: Xie, Ziwen, et al.
Published: (2026)

Prune-OPD: Efficient and Reliable On-Policy Distillation for Long-Horizon Reasoning
by: Yang, Zhicheng, et al.
Published: (2026)

Self-Distillation for Multi-Token Prediction
by: Zhao, Guoliang, et al.
Published: (2026)

Beyond Importance Sampling: Rejection-Gated Policy Optimization
by: Sun, Ziwu, et al.
Published: (2026)

IPD: Boosting Sequential Policy with Imaginary Planning Distillation in Offline Reinforcement Learning
by: Qin, Yihao, et al.
Published: (2026)

A Novel Hybrid Feature Importance and Feature Interaction Detection Framework for Predictive Optimization in Industry 4.0 Applications
by: Ma, Zhipeng, et al.
Published: (2024)

Do LLMs Encode Functional Importance of Reasoning Tokens?
by: Singh, Janvijay, et al.
Published: (2026)

HDPO: Hybrid Distillation Policy Optimization via Privileged Self-Distillation
by: Ding, Ken
Published: (2026)

Importance Analysis for Dynamic Control of Balancing Parameter in a Simple Knowledge Distillation Setting
by: Kim, Seongmin, et al.
Published: (2025)

OISD: On-Policy Internal Self-Distillation of Language Models
by: Liu, Xinyu, et al.
Published: (2026)