Image de couverture de livre

Post-Training is About States, Not Tokens: A State Distribution View of SFT, RL, and On-Policy Distillation

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteur principal:	Nie, Dong
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

GAC: Noise-Aware Adaptive Mixing for Hybrid SFT-RL Post-Training
par: Hu, Yuelin, et autres
Publié: (2026)

Quagmires in SFT-RL Post-Training: When High SFT Scores Mislead and What to Use Instead
par: Kang, Feiyang, et autres
Publié: (2025)

Explainable RL Policies by Distilling to Locally-Specialized Linear Policies with Voronoi State Partitioning
par: Deproost, Senne, et autres
Publié: (2025)

SFT-then-RL Outperforms Mixed-Policy Methods for LLM Reasoning
par: Limozin, Alexis, et autres
Publié: (2026)

Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillation
par: Yang, Zhuolin, et autres
Publié: (2026)

Trust the Batch, On- or Off-Policy: Adaptive Policy Optimization for RL Post-Training
par: Fakoor, Rasool, et autres
Publié: (2026)

RL Fine-Tuning Heals OOD Forgetting in SFT
par: Jin, Hangzhan, et autres
Publié: (2025)

Decoupling KL and Trajectories: A Unified Perspective for SFT, DAgger, Offline RL, and OPD in LLM Distillation
par: Zhao, Anhao, et autres
Publié: (2026)

Towards On-Policy SFT: Distribution Discriminant Theory and its Applications in LLM Training
par: Zhang, Miaosen, et autres
Publié: (2026)

SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training
par: Chu, Tianzhe, et autres
Publié: (2025)

TIP: Token Importance in On-Policy Distillation
par: Xu, Yuanda, et autres
Publié: (2026)

How to Compress KV Cache in RL Post-Training? Shadow Mask Distillation for Memory-Efficient Alignment
par: Zhu, Rui, et autres
Publié: (2026)

Consolidation or Adaptation? PRISM: Disentangling SFT and RL Data via Gradient Concentration
par: Zhao, Yang, et autres
Publié: (2026)

SFT-GRPO Data Overlap as a Post-Training Hyperparameter for Autoformalization
par: Su, Xiaole, et autres
Publié: (2026)

Action-Free Offline-to-Online RL via Discretised State Policies
par: Neggatu, Natinael Solomon, et autres
Publié: (2026)

Lightning OPD: Efficient Post-Training for Large Reasoning Models with Offline On-Policy Distillation
par: Wu, Yecheng, et autres
Publié: (2026)

$Q\sharp$: Provably Optimal Distributional RL for LLM Post-Training
par: Zhou, Jin Peng, et autres
Publié: (2025)

Large Language Model Post-Training: A Unified View of Off-Policy and On-Policy Learning
par: Zhao, Shiwan, et autres
Publié: (2026)

Imbalanced Gradients in RL Post-Training of Multi-Task LLMs
par: Wu, Runzhe, et autres
Publié: (2025)

Asymmetric On-Policy Distillation: Bridging Exploitation and Imitation at the Token Level
par: Jia, Nan, et autres
Publié: (2026)

Beyond GRPO and On-Policy Distillation: An Empirical Sparse-to-Dense Reward Principle for Language-Model Post-Training
par: Xu, Yuanda, et autres
Publié: (2026)

DVPO: Distributional Value Modeling-based Policy Optimization for LLM Post-Training
par: Zhu, Dingwei, et autres
Publié: (2025)

Critic-Driven Voronoi-Quantization for Distilling Deep RL Policies to Explainable Models
par: Deproost, Senne, et autres
Publié: (2026)

First SFT, Second RL, Third UPT: Continual Improving Multi-Modal LLM Reasoning via Unsupervised Post-Training
par: Wei, Lai, et autres
Publié: (2025)

Implicit Reward as the Bridge: A Unified View of SFT and DPO Connections
par: Wang, Bo, et autres
Publié: (2025)

DiRL: An Efficient Post-Training Framework for Diffusion Language Models
par: Zhu, Ying, et autres
Publié: (2025)

Quamba: A Post-Training Quantization Recipe for Selective State Space Models
par: Chiang, Hung-Yueh, et autres
Publié: (2024)

When Policies Cannot Be Retrained: A Unified Closed-Form View of Post-Training Steering in Offline Reinforcement Learning
par: Hossain, Elias, et autres
Publié: (2026)

mSFT: Addressing Dataset Mixtures Overfitting Heterogeneously in Multi-task SFT
par: Koh, Woosung, et autres
Publié: (2026)

Mechanistic origins of catastrophic forgetting: why RL preserves circuits better than SFT?
par: Nunez, Jeanmely Rojas, et autres
Publié: (2026)

Learning-Zone Energy: Online Data Selection for Efficient RL Post-Training
par: Cui, Peng, et autres
Publié: (2026)

CoScale-RL: Efficient Post-Training by Co-Scaling Data and Computation
par: Chen, Yutong, et autres
Publié: (2026)

AsyncFlow: An Asynchronous Streaming RL Framework for Efficient LLM Post-Training
par: Han, Zhenyu, et autres
Publié: (2025)

Bootstrapped Mixed Rewards for RL Post-Training: Injecting Canonical Action Order
par: Gupta, Prakhar, et autres
Publié: (2025)

When Are Teacher Tokens Reliable? Position-Weighted On-Policy Self-Distillation for Reasoning
par: Liu, Xiaogeng, et autres
Publié: (2026)

TRACE: Distilling Where It Matters via Token-Routed Self On-Policy Alignment
par: Wang, Jiaxuan, et autres
Publié: (2026)

Debunk the Myth of SFT Generalization
par: Lin, Xiaofeng, et autres
Publié: (2025)

Token-Efficient RL for LLM Reasoning
par: Lee, Alan, et autres
Publié: (2025)

Adversarial Latent-State Training for Robust Policies in Partially Observable Domains
par: Ahuja, Angad Singh
Publié: (2026)

AlignDistil: Token-Level Language Model Alignment as Adaptive Policy Distillation
par: Zhang, Songming, et autres
Publié: (2025)