Cover Image

Mind the Gap: Data Rewriting for Stable Off-Policy Supervised Fine-Tuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhao, Shiwan, Zhao, Xuyang, Zhou, Jiaming, Kong, Aobo, Li, Qicheng, Qin, Yong
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Large Language Model Post-Training: A Unified View of Off-Policy and On-Policy Learning
by: Zhao, Shiwan, et al.
Published: (2026)

Self-Prompt Tuning: Enable Autonomous Role-Playing in LLMs
by: Kong, Aobo, et al.
Published: (2024)

MAPEX: A Multi-Agent Pipeline for Keyphrase Extraction
by: Zhang, Liting, et al.
Published: (2025)

Enhancing Dysarthric Speech Recognition for Unseen Speakers via Prototype-Based Adaptation
by: Wang, Shiyao, et al.
Published: (2024)

RealTalk-CN: A Realistic Chinese Speech-Text Dialogue Benchmark With Cross-Modal Interaction Analysis
by: Wang, Enzhi, et al.
Published: (2025)

Patch the Distribution Mismatch: RL Rewriting Agent for Stable Off-Policy SFT
by: Wang, Jiacheng, et al.
Published: (2026)

Better Zero-Shot Reasoning with Role-Play Prompting
by: Kong, Aobo, et al.
Published: (2023)

Supervised Fine-Tuning Achieve Rapid Task Adaption Via Alternating Attention Head Activation Patterns
by: Zhao, Yang, et al.
Published: (2024)

EpilepsyLLM: Domain-Specific Large Language Model Fine-tuned with Epilepsy Medical Knowledge
by: Zhao, Xuyang, et al.
Published: (2024)

Towards Understanding the Fragility of Multilingual LLMs against Fine-Tuning Attacks
by: Poppi, Samuele, et al.
Published: (2024)

Anchored Supervised Fine-Tuning
by: Zhu, He, et al.
Published: (2025)

UFT: Unifying Supervised and Reinforcement Fine-Tuning
by: Liu, Mingyang, et al.
Published: (2025)

Proximal Supervised Fine-Tuning
by: Zhu, Wenhong, et al.
Published: (2025)

From Instance Selection to Fixed-Pool Data Recipe Search for Supervised Fine-Tuning
by: Wu, Haodong, et al.
Published: (2026)

SDPO: Segment-Level Direct Preference Optimization for Social Agents
by: Kong, Aobo, et al.
Published: (2025)

FisherSFT: Data-Efficient Supervised Fine-Tuning of Language Models Using Information Gain
by: Deb, Rohan, et al.
Published: (2025)

Triplets Better Than Pairs: Towards Stable and Effective Self-Play Fine-Tuning for LLMs
by: Wang, Yibo, et al.
Published: (2026)

Fine-Tuning LLMs for Report Summarization: Analysis on Supervised and Unsupervised Data
by: Rallapalli, Swati, et al.
Published: (2025)

Supervised Fine-Tuning Needs to Unlock the Potential of Token Priority
by: Shen, Zhanming, et al.
Published: (2026)

Parameter Importance is Not Static: Evolving Parameter Isolation for Supervised Fine-Tuning
by: Lin, Zekai, et al.
Published: (2026)

Supervised Fine-Tuning as Inverse Reinforcement Learning
by: Sun, Hao
Published: (2024)

Nested-ReFT: Efficient Reinforcement Learning for Large Language Model Fine-Tuning via Off-Policy Rollouts
by: Heuillet, Maxime, et al.
Published: (2025)

SRFT: A Single-Stage Method with Supervised and Reinforcement Fine-Tuning for Reasoning
by: Fu, Yuqian, et al.
Published: (2025)

Fine-Tuning Language Models with Reward Learning on Policy
by: Lang, Hao, et al.
Published: (2024)

From Parameters to Data: A Task-Parameter-Guided Fine-Tuning Pipeline for Efficient LLM Alignment
by: Chen, Hao, et al.
Published: (2026)

AirLLM: Diffusion Policy-based Adaptive LoRA for Remote Fine-Tuning of LLM over the Air
by: Yang, Shiyi, et al.
Published: (2025)

Blending Supervised and Reinforcement Fine-Tuning with Prefix Sampling
by: Huang, Zeyu, et al.
Published: (2025)

PL-FGSA: A Prompt Learning Framework for Fine-Grained Sentiment Analysis Based on MindSpore
by: Qin, Zhenkai, et al.
Published: (2025)

Pre-training LLM without Learning Rate Decay Enhances Supervised Fine-Tuning
by: Yano, Kazuki, et al.
Published: (2026)

Threshold Filtering Packing for Supervised Fine-Tuning: Training Related Samples within Packs
by: Dong, Jiancheng, et al.
Published: (2024)

Reflect-RL: Two-Player Online RL Fine-Tuning for LMs
by: Zhou, Runlong, et al.
Published: (2024)

SMoA: Spectrum Modulation Adapter for Parameter-Efficient Fine-Tuning
by: Liu, Yongkang, et al.
Published: (2026)

When Inverse Data Outperforms: Exploring the Pitfalls of Mixed Data in Multi-Stage Fine-Tuning
by: Deng, Mengyi, et al.
Published: (2025)

Mind the Privacy Unit! User-Level Differential Privacy for Language Model Fine-Tuning
by: Chua, Lynn, et al.
Published: (2024)

Entropy Polarity in Reinforcement Fine-Tuning: Direction, Asymmetry, and Control
by: Zhang, Jiazheng, et al.
Published: (2026)

PRewrite: Prompt Rewriting with Reinforcement Learning
by: Kong, Weize, et al.
Published: (2024)

Stabilizing LLM Supervised Fine-Tuning via Explicit Distributional Control
by: Wang, Xinyu, et al.
Published: (2026)

IA2: Alignment with ICL Activations Improves Supervised Fine-Tuning
by: Mishra, Aayush, et al.
Published: (2025)

R-Bot: An LLM-based Query Rewrite System
by: Sun, Zhaoyan, et al.
Published: (2024)

Causal Fine-Tuning under Latent Confounded Shift
by: Yu, Jialin, et al.
Published: (2024)