Image de couverture de livre

Quagmires in SFT-RL Post-Training: When High SFT Scores Mislead and What to Use Instead

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Kang, Feiyang, Kuchnik, Michael, Padthe, Karthik, Vlastelica, Marin, Jia, Ruoxi, Wu, Carole-Jean, Ardalani, Newsha
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Demystifying Synthetic Data in LLM Pre-training: A Systematic Study of Scaling Laws, Benefits, and Pitfalls
par: Kang, Feiyang, et autres
Publié: (2025)

GAC: Noise-Aware Adaptive Mixing for Hybrid SFT-RL Post-Training
par: Hu, Yuelin, et autres
Publié: (2026)

From SFT to RL: Demystifying the Post-Training Pipeline for LLM-based Vulnerability Detection
par: Li, Youpeng, et autres
Publié: (2026)

To 2:4 Sparsity and Beyond: Neuron-level Activation Function to Accelerate LLM Pre-Training
par: Madhyastha, Meghana, et autres
Publié: (2026)

When Does RL Help Medical VLMs? Disentangling Vision, SFT, and RL Gains
par: Jeddi, Ahmadreza, et autres
Publié: (2026)

The Synergy Dilemma of Long-CoT SFT and RL: Investigating Post-Training Techniques for Reasoning VLMs
par: Chen, Jierun, et autres
Publié: (2025)

Beyond Two-Stage Training: Cooperative SFT and RL for LLM Reasoning
par: Chen, Liang, et autres
Publié: (2025)

Post-Training is About States, Not Tokens: A State Distribution View of SFT, RL, and On-Policy Distillation
par: Nie, Dong
Publié: (2026)

RL makes MLLMs see better than SFT
par: Song, Junha, et autres
Publié: (2025)

RL Fine-Tuning Heals OOD Forgetting in SFT
par: Jin, Hangzhan, et autres
Publié: (2025)

Why Does RL Generalize Better Than SFT? A Data-Centric Perspective on VLM Post-Training
par: Lu, Aojun, et autres
Publié: (2026)

SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training
par: Chu, Tianzhe, et autres
Publié: (2025)

SFT-GRPO Data Overlap as a Post-Training Hyperparameter for Autoformalization
par: Su, Xiaole, et autres
Publié: (2026)

Text Quality-Based Pruning for Efficient Training of Language Models
par: Sharma, Vasu, et autres
Publié: (2024)

Bridging SFT and RL: Dynamic Policy Optimization for Robust Reasoning
par: Zhu, Taojie, et autres
Publié: (2026)

SFT-then-RL Outperforms Mixed-Policy Methods for LLM Reasoning
par: Limozin, Alexis, et autres
Publié: (2026)

mSFT: Addressing Dataset Mixtures Overfitting Heterogeneously in Multi-task SFT
par: Koh, Woosung, et autres
Publié: (2026)

First SFT, Second RL, Third UPT: Continual Improving Multi-Modal LLM Reasoning via Unsupervised Post-Training
par: Wei, Lai, et autres
Publié: (2025)

RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMs
par: Matsutani, Kohsei, et autres
Publié: (2025)

What Do Agents Learn from Trajectory-SFT: Semantics or Interfaces?
par: Gu, Weizheng, et autres
Publié: (2026)

DeReason: A Difficulty-Aware Curriculum Improves Decoupled SFT-then-RL Training for General Reasoning
par: Hu, Hanxu, et autres
Publié: (2026)

SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models
par: Chen, Hardy, et autres
Publié: (2025)

Debunk the Myth of SFT Generalization
par: Lin, Xiaofeng, et autres
Publié: (2025)

Beyond SFT-to-RL: Pre-alignment via Black-Box On-Policy Distillation for Multimodal RL
par: Wang, Sudong, et autres
Publié: (2026)

Patch the Distribution Mismatch: RL Rewriting Agent for Stable Off-Policy SFT
par: Wang, Jiacheng, et autres
Publié: (2026)

Consolidation or Adaptation? PRISM: Disentangling SFT and RL Data via Gradient Concentration
par: Zhao, Yang, et autres
Publié: (2026)

BREAD: Branched Rollouts from Expert Anchors Bridge SFT & RL for Reasoning
par: Zhang, Xuechen, et autres
Publié: (2025)

Metis-RISE: RL Incentivizes and SFT Enhances Multimodal Reasoning Model Learning
par: Qiu, Haibo, et autres
Publié: (2025)

Robust Post-Training for Generative Recommenders: Why Exponential Reward-Weighted SFT Outperforms RLHF
par: Chidambaram, Keertana, et autres
Publié: (2026)

Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyond
par: Wen, Liang, et autres
Publié: (2025)

Mechanistic origins of catastrophic forgetting: why RL preserves circuits better than SFT?
par: Nunez, Jeanmely Rojas, et autres
Publié: (2026)

Parallel-SFT: Improving Zero-Shot Cross-Programming-Language Transfer for Code RL
par: Wu, Zhaofeng, et autres
Publié: (2026)

OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cycles
par: Deng, Yihe, et autres
Publié: (2025)

How Much Backtracking is Enough? Exploring the Interplay of SFT and RL in Enhancing LLM Reasoning
par: Cai, Hongyi James, et autres
Publié: (2025)

SimpleAR: Pushing the Frontier of Autoregressive Visual Generation through Pretraining, SFT, and RL
par: Wang, Junke, et autres
Publié: (2025)

Crowd-SFT: Crowdsourcing for LLM Alignment
par: Sotiropoulos, Alex, et autres
Publié: (2025)

Getting More Juice Out of the SFT Data: Reward Learning from Human Demonstration Improves SFT for LLM Alignment
par: Li, Jiaxiang, et autres
Publié: (2024)

ReasonGen-R1: CoT for Autoregressive Image generation models through SFT and RL
par: Zhang, Yu, et autres
Publié: (2025)

A Three-Dimensional SFT with Sparse Columns
par: Salo, Ville, et autres
Publié: (2025)

Simplified SFT moduli spaces for Legendrian links
par: Avdek, Russell
Publié: (2021)