Cover Image

On-Policy RL Meets Off-Policy Experts: Harmonizing Supervised Fine-Tuning and Reinforcement Learning via Dynamic Weighting

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Wenhao, Xie, Yuexiang, Sun, Yuchang, Chen, Yanxi, Wang, Guoyin, Li, Yaliang, Ding, Bolin, Zhou, Jingren
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

On the Entropy Dynamics in Reinforcement Fine-Tuning of Large Language Models
by: Wang, Shumin, et al.
Published: (2026)

Exploring Selective Layer Fine-Tuning in Federated Learning
by: Sun, Yuchang, et al.
Published: (2024)

Trinity-RFT: A General-Purpose and Unified Framework for Reinforcement Fine-Tuning of Large Language Models
by: Pan, Xuchen, et al.
Published: (2025)

Group-Relative REINFORCE Is Secretly an Off-Policy Algorithm: Demystifying Some Myths About GRPO and Its Friends
by: Yao, Chaorui, et al.
Published: (2025)

EE-Tuning: An Economical yet Scalable Solution for Tuning Early-Exit Large Language Models
by: Pan, Xuchen, et al.
Published: (2024)

Enhancing Latent Computation in Transformers with Latent Tokens
by: Sun, Yuchang, et al.
Published: (2025)

Designing Algorithms Empowered by Language Models: An Analytical Framework, Case Studies, and Insights
by: Chen, Yanxi, et al.
Published: (2024)

Tree-based Models for Vertical Federated Learning: A Survey
by: Qian, Bingchen, et al.
Published: (2025)

EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelism
by: Chen, Yanxi, et al.
Published: (2023)

Provable Scaling Laws for the Test-Time Compute of Large Language Models
by: Chen, Yanxi, et al.
Published: (2024)

IntentRL: Training Proactive User-intent Agents for Open-ended Deep Research via Reinforcement Learning
by: Luo, Haohao, et al.
Published: (2026)

Mind the Gap: Data Rewriting for Stable Off-Policy Supervised Fine-Tuning
by: Zhao, Shiwan, et al.
Published: (2025)

Sparse but Critical: A Token-Level Analysis of Distributional Shifts in RLVR Fine-Tuning of LLMs
by: Meng, Haoming, et al.
Published: (2026)

Very Large-Scale Multi-Agent Simulation in AgentScope
by: Pan, Xuchen, et al.
Published: (2024)

Understanding Byzantine Robustness in Federated Learning with A Black-box Server
by: Zhao, Fangyuan, et al.
Published: (2024)

Agent-Oriented Planning in Multi-Agent Systems
by: Li, Ao, et al.
Published: (2024)

BOTS: A Unified Framework for Bayesian Online Task Selection in LLM Reinforcement Finetuning
by: Shen, Qianli, et al.
Published: (2025)

R$^3$L: Reflect-then-Retry Reinforcement Learning with Language-Guided Exploration, Pivotal Credit, and Positive Amplification
by: Shi, Weijie, et al.
Published: (2026)

FIPO: Eliciting Deep Reasoning with Future-KL Influenced Policy Optimization
by: Ma, Chiyu, et al.
Published: (2026)

Enhancing Multimodal Retrieval via Complementary Information Extraction and Alignment
by: Zeng, Delong, et al.
Published: (2026)

Policy Learning for Off-Dynamics RL with Deficient Support
by: Van, Linh Le Pham, et al.
Published: (2024)

SeeUPO: Sequence-Level Agentic-RL with Convergence Guarantees
by: Hu, Tianyi, et al.
Published: (2026)

On-Policy Supervised Fine-Tuning for Efficient Reasoning
by: Zhao, Anhao, et al.
Published: (2026)

Retrieval Heads are Dynamic
by: Lin, Yuping, et al.
Published: (2026)

Policy Agnostic RL: Offline RL and Online RL Fine-Tuning of Any Class and Backbone
by: Mark, Max Sobol, et al.
Published: (2024)

Data-Juicer Sandbox: A Feedback-Driven Suite for Multimodal Data-Model Co-development
by: Chen, Daoyuan, et al.
Published: (2024)

Automated Profile Inference with Language Model Agents
by: Du, Yuntao, et al.
Published: (2025)

Residual Off-Policy RL for Finetuning Behavior Cloning Policies
by: Ankile, Lars, et al.
Published: (2025)

Scaling Off-Policy Reinforcement Learning with Batch and Weight Normalization
by: Palenicek, Daniel, et al.
Published: (2025)

Nested-ReFT: Efficient Reinforcement Learning for Large Language Model Fine-Tuning via Off-Policy Rollouts
by: Heuillet, Maxime, et al.
Published: (2025)

Towards Anthropomorphic Conversational AI Part I: A Practical Framework
by: Wei, Fei, et al.
Published: (2025)

CRAFT: Counterfactual-to-Interactive Reinforcement Fine-Tuning for Driving Policies
by: Chen, Keyu, et al.
Published: (2026)

Soft Policy Optimization: Online Off-Policy RL for Sequence Models
by: Cohen, Taco, et al.
Published: (2025)

LLMs Can Learn to Reason Via Off-Policy RL
by: Ritter, Daniel, et al.
Published: (2026)

Improving LoRA in Privacy-preserving Federated Learning
by: Sun, Youbang, et al.
Published: (2024)

Supervised Fine-Tuning as Inverse Reinforcement Learning
by: Sun, Hao
Published: (2024)

IN-RIL: Interleaved Reinforcement and Imitation Learning for Policy Fine-Tuning
by: Gao, Dechen, et al.
Published: (2025)

From Off-Policy to On-Policy: Enhancing GUI Agents via Bi-level Expert-to-Policy Assimilation
by: Wang, Zezhou, et al.
Published: (2026)

FedBiOT: LLM Local Fine-tuning in Federated Learning without Full Model
by: Wu, Feijie, et al.
Published: (2024)

SCOPE-RL: A Python Library for Offline Reinforcement Learning and Off-Policy Evaluation
by: Kiyohara, Haruka, et al.
Published: (2023)