Copertina

Getting More Juice Out of the SFT Data: Reward Learning from Human Demonstration Improves SFT for LLM Alignment

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Li, Jiaxiang, Zeng, Siliang, Wai, Hoi-To, Li, Chenliang, Garcia, Alfredo, Hong, Mingyi
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Learning Reward and Policy Jointly from Demonstration and Preference Improves Alignment
di: Li, Chenliang, et al.
Pubblicazione: (2024)

When Demonstrations Meet Generative World Models: A Maximum Likelihood Framework for Offline Inverse Reinforcement Learning
di: Zeng, Siliang, et al.
Pubblicazione: (2023)

From Demonstrations to Rewards: Alignment Without Explicit Human Preferences
di: Zeng, Siliang, et al.
Pubblicazione: (2025)

RLHF in an SFT Way: From Optimal Solution to Reward-Weighted Alignment
di: Du, Yuhao, et al.
Pubblicazione: (2025)

Aligning Frozen LLMs by Reinforcement Learning: An Iterative Reweight-then-Optimize Approach
di: Zhang, Xinnan, et al.
Pubblicazione: (2025)

Structural Estimation of Markov Decision Processes in High-Dimensional State Space with Finite-Time Guarantees
di: Zeng, Siliang, et al.
Pubblicazione: (2022)

Implicit Reward as the Bridge: A Unified View of SFT and DPO Connections
di: Wang, Bo, et al.
Pubblicazione: (2025)

Understanding Inverse Reinforcement Learning under Overparameterization: Non-Asymptotic Analysis and Global Optimality
di: Zhang, Ruijia, et al.
Pubblicazione: (2025)

mSFT: Addressing Dataset Mixtures Overfitting Heterogeneously in Multi-task SFT
di: Koh, Woosung, et al.
Pubblicazione: (2026)

Crowd-SFT: Crowdsourcing for LLM Alignment
di: Sotiropoulos, Alex, et al.
Pubblicazione: (2025)

Debunk the Myth of SFT Generalization
di: Lin, Xiaofeng, et al.
Pubblicazione: (2025)

Quagmires in SFT-RL Post-Training: When High SFT Scores Mislead and What to Use Instead
di: Kang, Feiyang, et al.
Pubblicazione: (2025)

Minor SFT loss for LLM fine-tune to increase performance and reduce model deviation
di: Xie, Shiming, et al.
Pubblicazione: (2024)

A Bayesian Approach to Robust Inverse Reinforcement Learning
di: Wei, Ran, et al.
Pubblicazione: (2023)

SFT-then-RL Outperforms Mixed-Policy Methods for LLM Reasoning
di: Limozin, Alexis, et al.
Pubblicazione: (2026)

Stabilizing Off-Policy Training for Long-Horizon LLM Agent via Turn-Level Importance Sampling and Clipping-Triggered Normalization
di: Li, Chenliang, et al.
Pubblicazione: (2025)

From SFT to RL: Demystifying the Post-Training Pipeline for LLM-based Vulnerability Detection
di: Li, Youpeng, et al.
Pubblicazione: (2026)

Q-SFT: Q-Learning for Language Models via Supervised Fine-Tuning
di: Hong, Joey, et al.
Pubblicazione: (2024)

Improving LLM Safety and Helpfulness using SFT and DPO: A Study on OPT-350M
di: Pant, Piyush
Pubblicazione: (2025)

Towards On-Policy SFT: Distribution Discriminant Theory and its Applications in LLM Training
di: Zhang, Miaosen, et al.
Pubblicazione: (2026)

Does the Question Really Matter? Training-Free Data Selection for Vision-Language SFT
di: Sun, Peng, et al.
Pubblicazione: (2026)

PepThink-R1: LLM for Interpretable Cyclic Peptide Optimization with CoT SFT and Reinforcement Learning
di: Wang, Ruheng, et al.
Pubblicazione: (2025)

Consolidation or Adaptation? PRISM: Disentangling SFT and RL Data via Gradient Concentration
di: Zhao, Yang, et al.
Pubblicazione: (2026)

Breaking the SFT Plateau: Multimodal Structured Reinforcement Learning for Chart-to-Code Generation
di: Chen, Lei, et al.
Pubblicazione: (2025)

TEMPLE: Incentivizing Temporal Understanding of Video Large Language Models via Progressive Pre-SFT Alignment
di: Li, Shicheng, et al.
Pubblicazione: (2025)

Rethinking Generalization in Reasoning SFT: A Conditional Analysis on Optimization, Data, and Model Capability
di: Ren, Qihan, et al.
Pubblicazione: (2026)

How Much Backtracking is Enough? Exploring the Interplay of SFT and RL in Enhancing LLM Reasoning
di: Cai, Hongyi James, et al.
Pubblicazione: (2025)

Memory-Efficient LLM Pretraining via Minimalist Optimizer Design
di: Glentis, Athanasios, et al.
Pubblicazione: (2025)

PDC & DM-SFT: A Road for LLM SQL Bug-Fix Enhancing
di: Duan, Yiwen, et al.
Pubblicazione: (2024)

RL Fine-Tuning Heals OOD Forgetting in SFT
di: Jin, Hangzhan, et al.
Pubblicazione: (2025)

Memorize Theorems, Not Instances: Probing SFT Generalization through Mathematical Reasoning
di: Peng, Ruiying, et al.
Pubblicazione: (2026)

SFT-GRPO Data Overlap as a Post-Training Hyperparameter for Autoformalization
di: Su, Xiaole, et al.
Pubblicazione: (2026)

Reg-DPO: SFT-Regularized Direct Preference Optimization with GT-Pair for Improving Video Generation
di: Du, Jie, et al.
Pubblicazione: (2025)

Reconciling Contradictory Views on the Effectiveness of SFT in LLMs: An Interaction Perspective
di: Zhang, Junpeng, et al.
Pubblicazione: (2026)

Decoupling KL and Trajectories: A Unified Perspective for SFT, DAgger, Offline RL, and OPD in LLM Distillation
di: Zhao, Anhao, et al.
Pubblicazione: (2026)

GAC: Noise-Aware Adaptive Mixing for Hybrid SFT-RL Post-Training
di: Hu, Yuelin, et al.
Pubblicazione: (2026)

On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectification
di: Wu, Yongliang, et al.
Pubblicazione: (2025)

Gradients Must Earn Their Influence: Unifying SFT with Generalized Entropic Objectives
di: Wang, Zecheng, et al.
Pubblicazione: (2026)

First SFT, Second RL, Third UPT: Continual Improving Multi-Modal LLM Reasoning via Unsupervised Post-Training
di: Wei, Lai, et al.
Pubblicazione: (2025)

Reinforcing Multi-Turn Reasoning in LLM Agents via Turn-Level Reward Design
di: Wei, Quan, et al.
Pubblicazione: (2025)