Cover Image

RL Is Neither a Panacea Nor a Mirage: Understanding Supervised vs. Reinforcement Learning Fine-Tuning for LLMs

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Jin, Hangzhan, Lv, Sicheng, Wu, Sifan, Hamdaqa, Mohammad
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

RL Fine-Tuning Heals OOD Forgetting in SFT
by: Jin, Hangzhan, et al.
Published: (2025)

Rotation-Preserving Supervised Fine-Tuning
by: Jin, Hangzhan, et al.
Published: (2026)

CCCI: Code Completion with Contextual Information for Complex Data Transfer Tasks Using Large Language Models
by: Jin, Hangzhan, et al.
Published: (2025)

Modular Jets for Supervised Pipelines: Diagnosing Mirage vs Identifiability
by: Sanyal, Suman
Published: (2025)

RoPE Distinguishes Neither Positions Nor Tokens in Long Contexts, Provably
by: Du, Yufeng, et al.
Published: (2026)

On-Policy RL Meets Off-Policy Experts: Harmonizing Supervised Fine-Tuning and Reinforcement Learning via Dynamic Weighting
by: Zhang, Wenhao, et al.
Published: (2025)

Supervised Fine-Tuning as Inverse Reinforcement Learning
by: Sun, Hao
Published: (2024)

Supervised Fine Tuning on Curated Data is Reinforcement Learning (and can be improved)
by: Qin, Chongli, et al.
Published: (2025)

Understanding and Preserving Safety in Fine-Tuned LLMs
by: Zhang, Jiawen, et al.
Published: (2026)

Blending Supervised and Reinforcement Fine-Tuning with Prefix Sampling
by: Huang, Zeyu, et al.
Published: (2025)

Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-Tuning and Can Be Mitigated by Machine Unlearning
by: Chen, Yiwei, et al.
Published: (2025)

MIST-RL: Mutation-based Incremental Suite Testing via Reinforcement Learning
by: Zhu, Sicheng, et al.
Published: (2026)

Reinforcement Learning Fine-Tuning Enhances Activation Intensity and Diversity in the Internal Circuitry of LLMs
by: Zhang, Honglin, et al.
Published: (2025)

Efficient Differentially Private Fine-Tuning of LLMs via Reinforcement Learning
by: Khadangi, Afshin, et al.
Published: (2025)

Goal-Conditioned Supervised Learning for LLM Fine-Tuning
by: Li, Shijun, et al.
Published: (2026)

Policy Agnostic RL: Offline RL and Online RL Fine-Tuning of Any Class and Backbone
by: Mark, Max Sobol, et al.
Published: (2024)

RL-Struct: A Lightweight Reinforcement Learning Framework for Reliable Structured Output in LLMs
by: Hu, Ruike, et al.
Published: (2025)

GSQ-Tuning: Group-Shared Exponents Integer in Fully Quantized Training for LLMs On-Device Fine-tuning
by: Zhou, Sifan, et al.
Published: (2025)

Step-wise Adaptive Integration of Supervised Fine-tuning and Reinforcement Learning for Task-Specific LLMs
by: Chen, Jack, et al.
Published: (2025)

TruthRL: Incentivizing Truthful LLMs via Reinforcement Learning
by: Wei, Zhepei, et al.
Published: (2025)

Aligning LLMs with Biomedical Knowledge using Balanced Fine-Tuning
by: Tang, Zhenchao, et al.
Published: (2025)

Fine-Tuning LLMs for Report Summarization: Analysis on Supervised and Unsupervised Data
by: Rallapalli, Swati, et al.
Published: (2025)

Efficiency vs. Alignment: Investigating Safety and Fairness Risks in Parameter-Efficient Fine-Tuning of LLMs
by: Taraghi, Mina, et al.
Published: (2025)

Enhancing Reinforcement Learning Fine-Tuning with an Online Refiner
by: Ma, Hao, et al.
Published: (2026)

Robust Reinforcement Learning from Human Feedback for Large Language Models Fine-Tuning
by: Ye, Kai, et al.
Published: (2025)

Understanding Reinforcement Learning-Based Fine-Tuning of Diffusion Models: A Tutorial and Review
by: Uehara, Masatoshi, et al.
Published: (2024)

Proximal Supervised Fine-Tuning
by: Zhu, Wenhong, et al.
Published: (2025)

Reinforcement Learning Fine-Tunes a Sparse Subnetwork in Large Language Models
by: Balashov, Andrii
Published: (2025)

ESSAM: A Novel Competitive Evolution Strategies Approach to Reinforcement Learning for Memory Efficient LLMs Fine-Tuning
by: Sun, Zhishen, et al.
Published: (2026)

RL-BioAug: Label-Efficient Reinforcement Learning for Self-Supervised EEG Representation Learning
by: Lee, Cheol-Hui, et al.
Published: (2026)

Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs
by: Hübotter, Jonas, et al.
Published: (2024)

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
by: Wang, Xu, et al.
Published: (2025)

SRFT: A Single-Stage Method with Supervised and Reinforcement Fine-Tuning for Reasoning
by: Fu, Yuqian, et al.
Published: (2025)

Unlock the Correlation between Supervised Fine-Tuning and Reinforcement Learning in Training Code Large Language Models
by: Chen, Jie, et al.
Published: (2024)

Internalizing Curriculum Judgment for LLM Reinforcement Fine-Tuning
by: Zheng, Han, et al.
Published: (2026)

DipLLM: Fine-Tuning LLM for Strategic Decision-making in Diplomacy
by: Xu, Kaixuan, et al.
Published: (2025)

CoBA-RL: Capability-Oriented Budget Allocation for Reinforcement Learning in LLMs
by: Yao, Zhiyuan, et al.
Published: (2026)

Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens
by: Zhao, Chengshuai, et al.
Published: (2025)

RPO:Reinforcement Fine-Tuning with Partial Reasoning Optimization
by: Yi, Hongzhu, et al.
Published: (2026)

A Layer-wise Analysis of Supervised Fine-Tuning
by: Zhao, Qinghua, et al.
Published: (2026)