Image de couverture de livre

SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Chu, Tianzhe, Zhai, Yuexiang, Yang, Jihan, Tong, Shengbang, Xie, Saining, Schuurmans, Dale, Le, Quoc V., Levine, Sergey, Ma, Yi
Format:	Preprint
Publié:	2025
Sujets:	Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs
par: Tong, Shengbang, et autres
Publié: (2024)

Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning
par: Zhai, Yuexiang, et autres
Publié: (2024)

Diffusion Transformers with Representation Autoencoders
par: Zheng, Boyang, et autres
Publié: (2025)

White-Box Transformers via Sparse Rate Reduction: Compression Is All There Is?
par: Yu, Yaodong, et autres
Publié: (2023)

Seeing from Another Perspective: Evaluating Multi-View Understanding in MLLMs
par: Yeh, Chun-Hsiao, et autres
Publié: (2025)

RLIF: Interactive Imitation Learning as Reinforcement Learning
par: Luo, Jianlan, et autres
Publié: (2023)

Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuning
par: Nakamoto, Mitsuhiko, et autres
Publié: (2023)

Image Clustering via the Principle of Rate Reduction in the Age of Pretrained Models
par: Chu, Tianzhe, et autres
Publié: (2023)

Scaling Text-to-Image Diffusion Transformers with Representation Autoencoders
par: Tong, Shengbang, et autres
Publié: (2026)

RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMs
par: Matsutani, Kohsei, et autres
Publié: (2025)

DexHoldem: Playing Texas Hold'em with Dexterous Embodied System
par: Chen, Feng, et autres
Publié: (2026)

Quagmires in SFT-RL Post-Training: When High SFT Scores Mislead and What to Use Instead
par: Kang, Feiyang, et autres
Publié: (2025)

Q-SFT: Q-Learning for Language Models via Supervised Fine-Tuning
par: Hong, Joey, et autres
Publié: (2024)

Benchmark Designers Should "Train on the Test Set" to Expose Exploitable Non-Visual Shortcuts
par: Brown, Ellis, et autres
Publié: (2025)

Traveling Across Languages: Benchmarking Cross-Lingual Consistency in Multimodal LLMs
par: Wang, Hao, et autres
Publié: (2025)

V-IRL: Grounding Virtual Intelligence in Real Life
par: Yang, Jihan, et autres
Publié: (2024)

GAC: Noise-Aware Adaptive Mixing for Hybrid SFT-RL Post-Training
par: Hu, Yuelin, et autres
Publié: (2026)

From SFT to RL: Demystifying the Post-Training Pipeline for LLM-based Vulnerability Detection
par: Li, Youpeng, et autres
Publié: (2026)

Memorize Theorems, Not Instances: Probing SFT Generalization through Mathematical Reasoning
par: Peng, Ruiying, et autres
Publié: (2026)

RACER: Epistemic Risk-Sensitive RL Enables Fast Driving with Fewer Crashes
par: Stachowicz, Kyle, et autres
Publié: (2024)

The Synergy Dilemma of Long-CoT SFT and RL: Investigating Post-Training Techniques for Reasoning VLMs
par: Chen, Jierun, et autres
Publié: (2025)

Connecting Joint-Embedding Predictive Architecture with Contrastive Self-supervised Learning
par: Mo, Shentong, et autres
Publié: (2024)

Flow Map Distillation Without Data
par: Tong, Shangyuan, et autres
Publié: (2025)

Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs
par: Tong, Shengbang, et autres
Publié: (2024)

Post-Training is About States, Not Tokens: A State Distribution View of SFT, RL, and On-Policy Distillation
par: Nie, Dong
Publié: (2026)

METRA: Scalable Unsupervised RL with Metric-Aware Abstraction
par: Park, Seohong, et autres
Publié: (2023)

RL makes MLLMs see better than SFT
par: Song, Junha, et autres
Publié: (2025)

RL Fine-Tuning Heals OOD Forgetting in SFT
par: Jin, Hangzhan, et autres
Publié: (2025)

MetaMorph: Multimodal Understanding and Generation via Instruction Tuning
par: Tong, Shengbang, et autres
Publié: (2024)

Prioritized Replay for RL Post-training
par: Fatemi, Mehdi
Publié: (2026)

Spectral Ghost in Representation Learning: from Component Analysis to Self-Supervised Learning
par: Dai, Bo, et autres
Publié: (2026)

Autoregressive Large Language Models are Computationally Universal
par: Schuurmans, Dale, et autres
Publié: (2024)

Metis-RISE: RL Incentivizes and SFT Enhances Multimodal Reasoning Model Learning
par: Qiu, Haibo, et autres
Publié: (2025)

Why Does RL Generalize Better Than SFT? A Data-Centric Perspective on VLM Post-Training
par: Lu, Aojun, et autres
Publié: (2026)

Cambrian-S: Towards Spatial Supersensing in Video
par: Yang, Shusheng, et autres
Publié: (2025)

When Does RL Help Medical VLMs? Disentangling Vision, SFT, and RL Gains
par: Jeddi, Ahmadreza, et autres
Publié: (2026)

Planning without Search: Refining Frontier LLMs with Offline Goal-Conditioned RL
par: Hong, Joey, et autres
Publié: (2025)

Bridging SFT and RL: Dynamic Policy Optimization for Robust Reasoning
par: Zhu, Taojie, et autres
Publié: (2026)

SFT-then-RL Outperforms Mixed-Policy Methods for LLM Reasoning
par: Limozin, Alexis, et autres
Publié: (2026)

Decoupling KL and Trajectories: A Unified Perspective for SFT, DAgger, Offline RL, and OPD in LLM Distillation
par: Zhao, Anhao, et autres
Publié: (2026)