Copertina

Patch the Distribution Mismatch: RL Rewriting Agent for Stable Off-Policy SFT

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Wang, Jiacheng, Jian, Ping, Yang, Zhen, Chen, Zirong, Liao, Keren, Guo, Zhongbin
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Mind the Gap: Data Rewriting for Stable Off-Policy Supervised Fine-Tuning
di: Zhao, Shiwan, et al.
Pubblicazione: (2025)

SFT-then-RL Outperforms Mixed-Policy Methods for LLM Reasoning
di: Limozin, Alexis, et al.
Pubblicazione: (2026)

AceReason-Nemotron 1.1: Advancing Math and Code Reasoning through SFT and RL Synergy
di: Liu, Zihan, et al.
Pubblicazione: (2025)

Quagmires in SFT-RL Post-Training: When High SFT Scores Mislead and What to Use Instead
di: Kang, Feiyang, et al.
Pubblicazione: (2025)

Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyond
di: Wen, Liang, et al.
Pubblicazione: (2025)

Bridging SFT and RL: Dynamic Policy Optimization for Robust Reasoning
di: Zhu, Taojie, et al.
Pubblicazione: (2026)

QaRL: Rollout-Aligned Quantization-Aware RL for Fast and Stable Training under Training--Inference Mismatch
di: Gu, Hao, et al.
Pubblicazione: (2026)

Off-Policy Value-Based Reinforcement Learning for Large Language Models
di: Wang, Peng-Yuan, et al.
Pubblicazione: (2026)

Mechanistic origins of catastrophic forgetting: why RL preserves circuits better than SFT?
di: Nunez, Jeanmely Rojas, et al.
Pubblicazione: (2026)

Asynchronous RLHF: Faster and More Efficient Off-Policy RL for Language Models
di: Noukhovitch, Michael, et al.
Pubblicazione: (2024)

Reuse your FLOPs: Scaling RL on Hard Problems by Conditioning on Very Off-Policy Prefixes
di: Setlur, Amrith, et al.
Pubblicazione: (2026)

SPA-RL: Reinforcing LLM Agents via Stepwise Progress Attribution
di: Wang, Hanlin, et al.
Pubblicazione: (2025)

Missing Old Logits in Asynchronous Agentic RL: Semantic Mismatch and Repair Methods for Off-Policy Correction
di: Guan, Zhong, et al.
Pubblicazione: (2026)

Decoupling KL and Trajectories: A Unified Perspective for SFT, DAgger, Offline RL, and OPD in LLM Distillation
di: Zhao, Anhao, et al.
Pubblicazione: (2026)

Stable Asynchrony: Variance-Controlled Off-Policy RL for LLMs
di: Huang, Luke J., et al.
Pubblicazione: (2026)

On-Policy RL with Optimal Reward Baseline
di: Hao, Yaru, et al.
Pubblicazione: (2025)

Self-Augmented Preference Optimization: Off-Policy Paradigms for Language Model Alignment
di: Yin, Yueqin, et al.
Pubblicazione: (2024)

Jet-RL: Enabling On-Policy FP8 Reinforcement Learning with Unified Training and Rollout Precision Flow
di: Xi, Haocheng, et al.
Pubblicazione: (2026)

EdgeInfinite-Instruct: Bridging SFT-Based Optimization and NPU-Level Efficiency for Edge Devices
di: Chen, Jiyu, et al.
Pubblicazione: (2025)

Post-Training is About States, Not Tokens: A State Distribution View of SFT, RL, and On-Policy Distillation
di: Nie, Dong
Pubblicazione: (2026)

Memex(RL): Scaling Long-Horizon LLM Agents via Indexed Experience Memory
di: Wang, Zhenting, et al.
Pubblicazione: (2026)

Optimizing Large Language Models with an Enhanced LoRA Fine-Tuning Algorithm for Efficiency and Robustness in NLP Tasks
di: Hu, Jiacheng, et al.
Pubblicazione: (2024)

First SFT, Second RL, Third UPT: Continual Improving Multi-Modal LLM Reasoning via Unsupervised Post-Training
di: Wei, Lai, et al.
Pubblicazione: (2025)

Implicit Reward as the Bridge: A Unified View of SFT and DPO Connections
di: Wang, Bo, et al.
Pubblicazione: (2025)

EnvFactory: Scaling Tool-Use Agents via Executable Environments Synthesis and Robust RL
di: Xu, Minrui, et al.
Pubblicazione: (2026)

Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillation
di: Yang, Zhuolin, et al.
Pubblicazione: (2026)

BAPO: Stabilizing Off-Policy Reinforcement Learning for LLMs via Balanced Policy Optimization with Adaptive Clipping
di: Xi, Zhiheng, et al.
Pubblicazione: (2025)

DISA: Offline Importance Sampling for Distribution-Matching LLM-RL
di: Wang, Shaobo, et al.
Pubblicazione: (2026)

TMS: Trajectory-Mixed Supervision for Reward-Free, On-Policy SFT
di: Khan, Rana Muhammad Shahroz, et al.
Pubblicazione: (2026)

RubricEM: Meta-RL with Rubric-guided Policy Decomposition beyond Verifiable Rewards
di: Li, Gaotang, et al.
Pubblicazione: (2026)

Natural Language Actor-Critic: Scalable Off-Policy Learning in Language Space
di: Hong, Joey, et al.
Pubblicazione: (2025)

DynamixSFT: Dynamic Mixture Optimization of Instruction Tuning Collections
di: Shin, Haebin, et al.
Pubblicazione: (2025)

How Off-Policy Can GRPO Be? Mu-GRPO for Efficient LLM Reinforcement Learning
di: Tian, Minghao, et al.
Pubblicazione: (2026)

RAZOR: Sharpening Knowledge by Cutting Bias with Unsupervised Text Rewriting
di: Yang, Shuo, et al.
Pubblicazione: (2024)

The Struggle Between Continuation and Refusal: A Mechanistic Analysis of the Continuation-Triggered Jailbreak in LLMs
di: Deng, Yonghong, et al.
Pubblicazione: (2026)

Distributional Clarity: The Hidden Driver of RL-Friendliness in Large Language Models
di: Sun, Shaoning, et al.
Pubblicazione: (2026)

UFT: Unifying Fine-Tuning of SFT and RLHF/DPO/UNA through a Generalized Implicit Reward Function
di: Wang, Zhichao, et al.
Pubblicazione: (2024)

Learning to Reason under Off-Policy Guidance
di: Yan, Jianhao, et al.
Pubblicazione: (2025)

TreeRL: LLM Reinforcement Learning with On-Policy Tree Search
di: Hou, Zhenyu, et al.
Pubblicazione: (2025)

Gradients Must Earn Their Influence: Unifying SFT with Generalized Entropic Objectives
di: Wang, Zecheng, et al.
Pubblicazione: (2026)