Cover Image

SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chu, Tianzhe, Zhai, Yuexiang, Yang, Jihan, Tong, Shengbang, Xie, Saining, Schuurmans, Dale, Le, Quoc V., Levine, Sergey, Ma, Yi
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs
by: Tong, Shengbang, et al.
Published: (2024)

Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning
by: Zhai, Yuexiang, et al.
Published: (2024)

Diffusion Transformers with Representation Autoencoders
by: Zheng, Boyang, et al.
Published: (2025)

White-Box Transformers via Sparse Rate Reduction: Compression Is All There Is?
by: Yu, Yaodong, et al.
Published: (2023)

Seeing from Another Perspective: Evaluating Multi-View Understanding in MLLMs
by: Yeh, Chun-Hsiao, et al.
Published: (2025)

RLIF: Interactive Imitation Learning as Reinforcement Learning
by: Luo, Jianlan, et al.
Published: (2023)

Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuning
by: Nakamoto, Mitsuhiko, et al.
Published: (2023)

Image Clustering via the Principle of Rate Reduction in the Age of Pretrained Models
by: Chu, Tianzhe, et al.
Published: (2023)

Scaling Text-to-Image Diffusion Transformers with Representation Autoencoders
by: Tong, Shengbang, et al.
Published: (2026)

RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMs
by: Matsutani, Kohsei, et al.
Published: (2025)

DexHoldem: Playing Texas Hold'em with Dexterous Embodied System
by: Chen, Feng, et al.
Published: (2026)

Quagmires in SFT-RL Post-Training: When High SFT Scores Mislead and What to Use Instead
by: Kang, Feiyang, et al.
Published: (2025)

Q-SFT: Q-Learning for Language Models via Supervised Fine-Tuning
by: Hong, Joey, et al.
Published: (2024)

Benchmark Designers Should "Train on the Test Set" to Expose Exploitable Non-Visual Shortcuts
by: Brown, Ellis, et al.
Published: (2025)

Traveling Across Languages: Benchmarking Cross-Lingual Consistency in Multimodal LLMs
by: Wang, Hao, et al.
Published: (2025)

V-IRL: Grounding Virtual Intelligence in Real Life
by: Yang, Jihan, et al.
Published: (2024)

GAC: Noise-Aware Adaptive Mixing for Hybrid SFT-RL Post-Training
by: Hu, Yuelin, et al.
Published: (2026)

From SFT to RL: Demystifying the Post-Training Pipeline for LLM-based Vulnerability Detection
by: Li, Youpeng, et al.
Published: (2026)

Memorize Theorems, Not Instances: Probing SFT Generalization through Mathematical Reasoning
by: Peng, Ruiying, et al.
Published: (2026)

RACER: Epistemic Risk-Sensitive RL Enables Fast Driving with Fewer Crashes
by: Stachowicz, Kyle, et al.
Published: (2024)

The Synergy Dilemma of Long-CoT SFT and RL: Investigating Post-Training Techniques for Reasoning VLMs
by: Chen, Jierun, et al.
Published: (2025)

Connecting Joint-Embedding Predictive Architecture with Contrastive Self-supervised Learning
by: Mo, Shentong, et al.
Published: (2024)

Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs
by: Tong, Shengbang, et al.
Published: (2024)

Flow Map Distillation Without Data
by: Tong, Shangyuan, et al.
Published: (2025)

Post-Training is About States, Not Tokens: A State Distribution View of SFT, RL, and On-Policy Distillation
by: Nie, Dong
Published: (2026)

METRA: Scalable Unsupervised RL with Metric-Aware Abstraction
by: Park, Seohong, et al.
Published: (2023)

RL makes MLLMs see better than SFT
by: Song, Junha, et al.
Published: (2025)

RL Fine-Tuning Heals OOD Forgetting in SFT
by: Jin, Hangzhan, et al.
Published: (2025)

MetaMorph: Multimodal Understanding and Generation via Instruction Tuning
by: Tong, Shengbang, et al.
Published: (2024)

Prioritized Replay for RL Post-training
by: Fatemi, Mehdi
Published: (2026)

Spectral Ghost in Representation Learning: from Component Analysis to Self-Supervised Learning
by: Dai, Bo, et al.
Published: (2026)

Autoregressive Large Language Models are Computationally Universal
by: Schuurmans, Dale, et al.
Published: (2024)

Metis-RISE: RL Incentivizes and SFT Enhances Multimodal Reasoning Model Learning
by: Qiu, Haibo, et al.
Published: (2025)

Cambrian-S: Towards Spatial Supersensing in Video
by: Yang, Shusheng, et al.
Published: (2025)

Why Does RL Generalize Better Than SFT? A Data-Centric Perspective on VLM Post-Training
by: Lu, Aojun, et al.
Published: (2026)

When Does RL Help Medical VLMs? Disentangling Vision, SFT, and RL Gains
by: Jeddi, Ahmadreza, et al.
Published: (2026)

Planning without Search: Refining Frontier LLMs with Offline Goal-Conditioned RL
by: Hong, Joey, et al.
Published: (2025)

Bridging SFT and RL: Dynamic Policy Optimization for Robust Reasoning
by: Zhu, Taojie, et al.
Published: (2026)

SFT-then-RL Outperforms Mixed-Policy Methods for LLM Reasoning
by: Limozin, Alexis, et al.
Published: (2026)

Decoupling KL and Trajectories: A Unified Perspective for SFT, DAgger, Offline RL, and OPD in LLM Distillation
by: Zhao, Anhao, et al.
Published: (2026)