Cover Image

One-Token Rollout: Guiding Supervised Fine-Tuning of LLMs with Policy Gradient

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ming, Rui, Wu, Haoyuan, Hu, Shoubo, He, Zhuolun, Yu, Bei
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

ToTRL: Unlock LLM Tree-of-Thoughts Reasoning Potential through Puzzles Solving
by: Wu, Haoyuan, et al.
Published: (2025)

Divergent Thoughts toward One Goal: LLM-based Multi-Agent Collaboration System for Electronic Design Automation
by: Wu, Haoyuan, et al.
Published: (2025)

Efficient OpAmp Adaptation for Zoom Attention to Golden Contexts
by: Wu, Haoyuan, et al.
Published: (2025)

Architect of the Bits World: Masked Autoregressive Modeling for Circuit Generation Guided by Truth Table
by: Wu, Haoyuan, et al.
Published: (2025)

On-Policy Optimization with Group Equivalent Preference for Multi-Programming Language Understanding
by: Wu, Haoyuan, et al.
Published: (2025)

Customized Retrieval Augmented Generation and Benchmarking for EDA Tool Documentation QA
by: Pu, Yuan, et al.
Published: (2024)

Rethinking Data Selection for Supervised Fine-Tuning
by: Shen, Ming
Published: (2024)

Supervised Fine-Tuning Needs to Unlock the Potential of Token Priority
by: Shen, Zhanming, et al.
Published: (2026)

Parameter-Efficient Sparsity Crafting from Dense to Mixture-of-Experts for Instruction Tuning on General Tasks
by: Wu, Haoyuan, et al.
Published: (2024)

Token Cleaning: Fine-Grained Data Selection for LLM Supervised Fine-Tuning
by: Pang, Jinlong, et al.
Published: (2025)

Semantic Loss Guided Data Efficient Supervised Fine Tuning for Safe Responses in LLMs
by: Lu, Yuxiao, et al.
Published: (2024)

Towards Pedagogical LLMs with Supervised Fine Tuning for Computing Education
by: Vassar, Alexandra, et al.
Published: (2024)

Beyond Negative Rollouts: Positive-Only Policy Optimization with Implicit Negative Gradients
by: Xu, Mingwei, et al.
Published: (2026)

Diversity or Precision? A Deep Dive into Next Token Prediction
by: Wu, Haoyuan, et al.
Published: (2025)

GradPruner: Gradient-Guided Layer Pruning Enabling Efficient Fine-Tuning and Inference for LLMs
by: Huang, Wei, et al.
Published: (2026)

Anchored Supervised Fine-Tuning
by: Zhu, He, et al.
Published: (2025)

Supervised Fine-Tuning LLMs to Behave as Pedagogical Agents in Programming Education
by: Ross, Emily, et al.
Published: (2025)

Proximal Supervised Fine-Tuning
by: Zhu, Wenhong, et al.
Published: (2025)

Are Full Rollouts Necessary for On-Policy Distillation?
by: Zhang, Yaocheng, et al.
Published: (2026)

Mitigating Forgetting in LLM Fine-Tuning via Low-Perplexity Token Learning
by: Wu, Chao-Chung, et al.
Published: (2025)

Nested-ReFT: Efficient Reinforcement Learning for Large Language Model Fine-Tuning via Off-Policy Rollouts
by: Heuillet, Maxime, et al.
Published: (2025)

Analyzing the Effects of Supervised Fine-Tuning on Model Knowledge from Token and Parameter Levels
by: Ye, Junjie, et al.
Published: (2025)

Rethinking Supervised Fine-Tuning: Emphasizing Key Answer Tokens for Improved LLM Accuracy
by: Shi, Xiaofeng, et al.
Published: (2025)

Supervised Fine-Tuning or In-Context Learning? Evaluating LLMs for Clinical NER
by: Baroian, Andrei
Published: (2025)

Unleashing the Reasoning Potential of Pre-trained LLMs by Critique Fine-Tuning on One Problem
by: Wang, Yubo, et al.
Published: (2025)

Token Buncher: Shielding LLMs from Harmful Reinforcement Learning Fine-Tuning
by: Feng, Weitao, et al.
Published: (2025)

Fine-Tuning Discrete Diffusion Models with Policy Gradient Methods
by: Zekri, Oussama, et al.
Published: (2025)

FineMedLM-o1: Enhancing Medical Knowledge Reasoning Ability of LLM from Supervised Fine-Tuning to Test-Time Training
by: Yu, Hongzhou, et al.
Published: (2025)

Winning the Pruning Gamble: A Unified Approach to Joint Sample and Token Pruning for Efficient Supervised Fine-Tuning
by: Wang, Shaobo, et al.
Published: (2025)

Mind the Gap: Data Rewriting for Stable Off-Policy Supervised Fine-Tuning
by: Zhao, Shiwan, et al.
Published: (2025)

From Instance Selection to Fixed-Pool Data Recipe Search for Supervised Fine-Tuning
by: Wu, Haodong, et al.
Published: (2026)

Gradient Surgery for Safe LLM Fine-Tuning
by: Yi, Biao, et al.
Published: (2025)

Fine-Tuning LLMs for Report Summarization: Analysis on Supervised and Unsupervised Data
by: Rallapalli, Swati, et al.
Published: (2025)

SFT-TA: Supervised Fine-Tuned Agents in Multi-Agent LLMs for Automated Inductive Thematic Analysis
by: Yi, Seungjun, et al.
Published: (2025)

FocuSFT: Bilevel Optimization for Dilution-Aware Long-Context Fine-Tuning
by: Pei, Zehua, et al.
Published: (2026)

Reassessing the Role of Supervised Fine-Tuning: An Empirical Study in VLM Reasoning
by: Yu, Yongcan, et al.
Published: (2025)

Enhancing Large Language Model Reasoning via Selective Critical Token Fine-Tuning
by: Ruan, Zhiwen, et al.
Published: (2025)

Open-Source LLMs for Text Annotation: A Practical Guide for Model Setting and Fine-Tuning
by: Alizadeh, Meysam, et al.
Published: (2023)

Supervised In-Context Fine-Tuning for Generative Sequence Labeling
by: Dukić, David, et al.
Published: (2025)

ResT: Reshaping Token-Level Policy Gradients for Tool-Use Large Language Models
by: Lin, Zihan, et al.
Published: (2025)