Cover Image

Are Full Rollouts Necessary for On-Policy Distillation?

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Yaocheng, Chai, Jiajun, Fu, Yuqian, Tu, Songjun, Wang, Xiaohan, Lin, Wei, Yin, Guojun, Zhang, Qichao, Zhu, Yuanheng, Zhao, Dongbin
Format:	Preprint
Published:	2026
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

$π$-Play: Multi-Agent Self-Play via Privileged Self-Distillation without External Data
by: Zhang, Yaocheng, et al.
Published: (2026)

SRFT: A Single-Stage Method with Supervised and Reinforcement Fine-Tuning for Reasoning
by: Fu, Yuqian, et al.
Published: (2025)

RLAE: Reinforcement Learning-Assisted Ensemble for LLMs
by: Fu, Yuqian, et al.
Published: (2025)

AutoSearch: Adaptive Search Depth for Efficient Agentic RAG via Reinforcement Learning
by: Sun, Jingbo, et al.
Published: (2026)

CriticSearch: Fine-Grained Credit Assignment for Search Agents via a Retrospective Critic
by: Zhang, Yaocheng, et al.
Published: (2025)

Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes
by: Fu, Yuqian, et al.
Published: (2026)

CPIG: Leveraging Consistency Policy with Intention Guidance for Multi-agent Exploration
by: Fu, Yuqian, et al.
Published: (2024)

Enhancing LLM Reasoning with Iterative DPO: A Comprehensive Empirical Investigation
by: Tu, Songjun, et al.
Published: (2025)

ResT: Reshaping Token-Level Policy Gradients for Tool-Use Large Language Models
by: Lin, Zihan, et al.
Published: (2025)

In-Dataset Trajectory Return Regularization for Offline Preference-based Reinforcement Learning
by: Tu, Songjun, et al.
Published: (2024)

Salience-Invariant Consistent Policy Learning for Generalization in Visual Reinforcement Learning
by: Sun, Jingbo, et al.
Published: (2025)

Learning When to Think: Shaping Adaptive Reasoning in R1-Style Models via Multi-Stage RL
by: Tu, Songjun, et al.
Published: (2025)

DipLLM: Fine-Tuning LLM for Strategic Decision-making in Diplomacy
by: Xu, Kaixuan, et al.
Published: (2025)

Perception-Consistency Multimodal Large Language Models Reasoning via Caption-Regularized Policy Optimization
by: Tu, Songjun, et al.
Published: (2025)

Dynamic Dual-Granularity Skill Bank for Agentic RL
by: Tu, Songjun, et al.
Published: (2026)

Implicit Hierarchical GRPO: Decoupling Tool Invocation from Execution for Tool-Integrated Mathematical Reasoning
by: Wang, Li, et al.
Published: (2026)

Online Preference-based Reinforcement Learning with Self-augmented Feedback from Large Language Model
by: Tu, Songjun, et al.
Published: (2024)

PaperAudit-Bench: Benchmarking Error Detection in Research Papers for Critical Automated Peer Review
by: Tu, Songjun, et al.
Published: (2026)

AWPO: Enhancing Tool-Use of Large Language Models through Adaptive Integration of Reasoning Rewards
by: Lin, Zihan, et al.
Published: (2025)

When Self-Belief Misleads: Active Label Acquisition for Reinforcement Learning with Verifiable Rewards
by: Wang, Li, et al.
Published: (2026)

Rethinking Personalization in Large Language Models at the Token Level
by: Zhang, Chenheng, et al.
Published: (2026)

ResRL: Boosting LLM Reasoning via Negative Sample Projection Residual Reinforcement Learning
by: Lin, Zihan, et al.
Published: (2026)

From Experience to Strategy: Empowering LLM Agents with Trainable Graph Memory
by: Xia, Siyu, et al.
Published: (2025)

Contextual Rollout Bandits for Reinforcement Learning with Verifiable Rewards
by: Lu, Xiaodong, et al.
Published: (2026)

Saliency-Guided Representation with Consistency Policy Learning for Visual Unsupervised Reinforcement Learning
by: Sun, Jingbo, et al.
Published: (2026)

A Hierarchical Deep Reinforcement Learning Framework for 6-DOF UCAV Air-to-Air Combat
by: Chai, Jiajun, et al.
Published: (2022)

AMR-SD: Asymmetric Meta-Reflective Self-Distillation for Token-Level Credit Assignment
by: Wei, Zhenlin, et al.
Published: (2026)

Discretizing Continuous Action Space with Unimodal Probability Distributions for On-Policy Reinforcement Learning
by: Zhu, Yuanyang, et al.
Published: (2024)

SAE as a Crystal Ball: Interpretable Features Predict Cross-domain Transferability of LLMs without Training
by: Zhang, Qi, et al.
Published: (2026)

On-Policy Distillation with Best-of-N Teacher Rollout Selection
by: Zhang, Ke, et al.
Published: (2026)

Joint Training of Multi-Token Prediction in Reinforcement Learning via Optimal Coefficient Calibration
by: Wang, Zili, et al.
Published: (2026)

SPEC-RL: Accelerating On-Policy Reinforcement Learning with Speculative Rollouts
by: Liu, Bingshuai, et al.
Published: (2025)

Jet-RL: Enabling On-Policy FP8 Reinforcement Learning with Unified Training and Rollout Precision Flow
by: Xi, Haocheng, et al.
Published: (2026)

ZipRL: Adaptive Multi-Turn Context Compression with Hindsight Response Replay
by: Hu, Zhexin, et al.
Published: (2026)

Spec-o3: A Tool-Augmented Vision-Language Agent for Rare Celestial Object Candidate Vetting via Automated Spectral Inspection
by: Jia, Minghui, et al.
Published: (2026)

Semantic Convergence: Harmonizing Recommender Systems via Two-Stage Alignment and Behavioral Semantic Tokenization
by: Li, Guanghan, et al.
Published: (2024)

Enhancing Knowledge Distillation of Large Language Models through Efficient Multi-Modal Distribution Alignment
by: Peng, Tianyu, et al.
Published: (2024)

A General Highly Accurate Online Planning Method Integrating Large Language Models into Nested Rollout Policy Adaptation for Dialogue Tasks
by: Wang, Hui, et al.
Published: (2025)

CLIPO: Contrastive Learning in Policy Optimization Generalizes RLVR
by: Cui, Sijia, et al.
Published: (2026)

ARAC: Adaptive Regularized Multi-Agent Soft Actor-Critic in Graph-Structured Adversarial Games
by: Shi, Ruochuan, et al.
Published: (2025)