Cover Image

Scalpel vs. Hammer: GRPO Amplifies Existing Capabilities, SFT Replaces Them

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Rajani, Neel, Gema, Aryo Pradipta, Goldfarb-Tarrant, Seraphina, Titov, Ivan
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Safer or Luckier? LLMs as Safety Evaluators Are Not Robust to Artifacts
by: Chen, Hongyu, et al.
Published: (2025)

The Multilingual Alignment Prism: Aligning Global and Local Preferences to Reduce Harm
by: Aakanksha, et al.
Published: (2024)

CoMAT: Chain of Mathematically Annotated Thought Improves Mathematical Reasoning
by: Leang, Joshua Ong Jun, et al.
Published: (2024)

Large Language Models Generate Harmful Content Using a Distinct, Unified Mechanism
by: Orgad, Hadas, et al.
Published: (2026)

Learning is Forgetting: LLM Training As Lossy Compression
by: Conklin, Henry C., et al.
Published: (2026)

Lost in Simulation: LLM-Simulated Users are Unreliable Proxies for Human Users in Agentic Evaluations
by: Seshadri, Preethi, et al.
Published: (2026)

MultiContrievers: Analysis of Dense Retrieval Representations
by: Goldfarb-Tarrant, Seraphina, et al.
Published: (2024)

Lost in Time: Clock and Calendar Understanding Challenges in Multimodal LLMs
by: Saxena, Rohit, et al.
Published: (2025)

What's New in My Data? Novelty Exploration via Contrastive Generation
by: Isonuma, Masaru, et al.
Published: (2024)

Mix Data or Merge Models? Optimizing for Diverse Multi-Task Learning
by: Aakanksha, et al.
Published: (2024)

Shared Doubt: Zero-shot Cross-Lingual Confidence Estimation for Language Models
by: Kyriakou, Athina, et al.
Published: (2026)

Joint Localization and Activation Editing for Low-Resource Fine-Tuning
by: Lai, Wen, et al.
Published: (2025)

Quagmires in SFT-RL Post-Training: When High SFT Scores Mislead and What to Use Instead
by: Kang, Feiyang, et al.
Published: (2025)

Parameter-Efficient Fine-Tuning of LLaMA for the Clinical Domain
by: Gema, Aryo Pradipta, et al.
Published: (2023)

Finding Culture-Sensitive Neurons in Vision-Language Models
by: Zhao, Xiutian, et al.
Published: (2025)

SFT-then-RL Outperforms Mixed-Policy Methods for LLM Reasoning
by: Limozin, Alexis, et al.
Published: (2026)

DynamixSFT: Dynamic Mixture Optimization of Instruction Tuning Collections
by: Shin, Haebin, et al.
Published: (2025)

Edinburgh Clinical NLP at MEDIQA-CORR 2024: Guiding Large Language Models with Hints
by: Gema, Aryo Pradipta, et al.
Published: (2024)

The Rogue Scalpel: Activation Steering Compromises LLM Safety
by: Korznikov, Anton, et al.
Published: (2025)

ExGRPO: Learning to Reason from Experience
by: Zhan, Runzhe, et al.
Published: (2025)

Gradients Must Earn Their Influence: Unifying SFT with Generalized Entropic Objectives
by: Wang, Zecheng, et al.
Published: (2026)

Implicit Reward as the Bridge: A Unified View of SFT and DPO Connections
by: Wang, Bo, et al.
Published: (2025)

RLHF in an SFT Way: From Optimal Solution to Reward-Weighted Alignment
by: Du, Yuhao, et al.
Published: (2025)

SFT-GRPO Data Overlap as a Post-Training Hyperparameter for Autoformalization
by: Su, Xiaole, et al.
Published: (2026)

Q-SFT: Q-Learning for Language Models via Supervised Fine-Tuning
by: Hong, Joey, et al.
Published: (2024)

PDC & DM-SFT: A Road for LLM SQL Bug-Fix Enhancing
by: Duan, Yiwen, et al.
Published: (2024)

Can GRPO Help LLMs Transcend Their Pretraining Origin?
by: Ni, Kangqi, et al.
Published: (2025)

From Reasoning to Code: GRPO Optimization for Underrepresented Languages
by: Pennino, Federico, et al.
Published: (2025)

Balanced Aggregation: Understanding and Fixing Aggregation Bias in GRPO
by: Zeng, Zhiyuan, et al.
Published: (2026)

A Toolbox, Not a Hammer -- Multi-TAG: Scaling Math Reasoning with Multi-Tool Aggregation
by: Yao, Bohan, et al.
Published: (2025)

Amplifying, Not Learning: Fine-Tuned AI Text Detectors Amplify a Pretrained Direction
by: Smirnov, Alexander
Published: (2026)

Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Data
by: Wang, Xinyi, et al.
Published: (2024)

Mechanistic origins of catastrophic forgetting: why RL preserves circuits better than SFT?
by: Nunez, Jeanmely Rojas, et al.
Published: (2026)

Climbing the Ladder of Reasoning: What LLMs Can-and Still Can't-Solve after SFT?
by: Sun, Yiyou, et al.
Published: (2025)

Multi-Task GRPO: Reliable LLM Reasoning Across Tasks
by: Ramesh, Shyam Sundhar, et al.
Published: (2026)

MMR-GRPO: Accelerating GRPO-Style Training through Diversity-Aware Reward Reweighting
by: Wei, Kangda, et al.
Published: (2026)

Improving LLM Safety and Helpfulness using SFT and DPO: A Study on OPT-350M
by: Pant, Piyush
Published: (2025)

Delving into RL for Image Generation with CoT: A Study on DPO vs. GRPO
by: Tong, Chengzhuo, et al.
Published: (2025)

MiGrATe: Mixed-Policy GRPO for Adaptation at Test-Time
by: Phan, Peter, et al.
Published: (2025)

Stepwise Guided Policy Optimization: Coloring your Incorrect Reasoning in GRPO
by: Chen, Peter, et al.
Published: (2025)