Cover Image

Optimistic Model Rollouts for Pessimistic Offline Policy Optimization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhai, Yuanzhao, Li, Yiying, Gao, Zijian, Gong, Xudong, Xu, Kele, Feng, Dawei, Bo, Ding, Wang, Huaimin
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Uncertainty-Penalized Reinforcement Learning from Human Feedback with Diverse Reward LoRA Ensembles
by: Zhai, Yuanzhao, et al.
Published: (2023)

Online Self-Preferring Language Models
by: Zhai, Yuanzhao, et al.
Published: (2024)

Enhancing Decision-Making for LLM Agents via Step-Level Q-Value Models
by: Zhai, Yuanzhao, et al.
Published: (2024)

Optimistic Policy Learning under Pessimistic Adversaries with Regret and Violation Guarantees
by: Ganguly, Sourav, et al.
Published: (2026)

MAny: Merge Anything for Multimodal Continual Instruction Tuning
by: Gao, Zijian, et al.
Published: (2026)

Scaffold-Conditioned Preference Triplets for Controllable Molecular Optimization with Large Language Models
by: Xiong, Yi, et al.
Published: (2026)

Diverse Randomized Value Functions: A Provably Pessimistic Approach for Offline Reinforcement Learning
by: Yu, Xudong, et al.
Published: (2024)

Hyperparameter Tuning Through Pessimistic Bilevel Optimization
by: Ustun, Meltem Apaydin, et al.
Published: (2024)

Pessimistic Causal Reinforcement Learning with Mediators for Confounded Offline Data
by: Wang, Danyang, et al.
Published: (2024)

Pessimistic Off-Policy Optimization for Learning to Rank
by: Cief, Matej, et al.
Published: (2022)

Pessimistic Backward Policy for GFlowNets
by: Jang, Hyosoon, et al.
Published: (2024)

Pessimistic Value Iteration for Multi-Task Data Sharing in Offline Reinforcement Learning
by: Bai, Chenjia, et al.
Published: (2024)

Pessimistic Risk-Aware Policy Learning in Contextual Bandits
by: Wan, Yilong, et al.
Published: (2026)

Optimistic Policy Optimization is Provably Efficient in Non-stationary MDPs
by: Zhong, Han, et al.
Published: (2021)

Diffusion World Model: Future Modeling Beyond Step-by-Step Rollout for Offline Reinforcement Learning
by: Ding, Zihan, et al.
Published: (2024)

Feasibility-Aware Pessimistic Estimation: Toward Long-Horizon Safety in Offline RL
by: Tao, Zhikun
Published: (2025)

Pessimistic Nonlinear Least-Squares Value Iteration for Offline Reinforcement Learning
by: Di, Qiwei, et al.
Published: (2023)

Affective Music Recommendation: A Rollout-Based World Model for Offline Preference Optimization
by: Chan, Audrey, et al.
Published: (2026)

COPR: Continual Learning Human Preference through Optimal Policy Regularization
by: Zhang, Han, et al.
Published: (2023)

Optimistic Policy Regularization
by: Pham, Mai, et al.
Published: (2026)

Learning a Pessimistic Reward Model in RLHF
by: Xu, Yinglun, et al.
Published: (2025)

Where to Spend Rollouts: Hit-Utility Optimal Rollout Allocation for Group-Based RLVR
by: Wang, Tao, et al.
Published: (2026)

Optimistic Critic Reconstruction and Constrained Fine-Tuning for General Offline-to-Online RL
by: Luo, Qin-Wen, et al.
Published: (2024)

Selective Rollout: Mid-Trajectory Termination for Multi-Sample Agent RL
by: Zhai, Zhiyuan, et al.
Published: (2026)

Logarithmic Smoothing for Pessimistic Off-Policy Evaluation, Selection and Learning
by: Sakhi, Otmane, et al.
Published: (2024)

Horizon Imagination: Efficient On-Policy Rollout in Diffusion World Models
by: Cohen, Lior, et al.
Published: (2026)

CROP: Conservative Reward for Model-based Offline Policy Optimization
by: Li, Hao, et al.
Published: (2023)

Optimistic Multi-Agent Policy Gradient
by: Zhao, Wenshuai, et al.
Published: (2023)

POLAR: A Pessimistic Model-based Policy Learning Algorithm for Dynamic Treatment Regimes
by: Zhang, Ruijia, et al.
Published: (2025)

Breaking the Curse of Repulsion: Optimistic Distributionally Robust Policy Optimization for Off-Policy Generative Recommendation
by: Jiang, Jie, et al.
Published: (2026)

Knowledgeable Agents by Offline Reinforcement Learning from Large Language Model Rollouts
by: Pang, Jing-Cheng, et al.
Published: (2024)

Fat-to-Thin Policy Optimization: Offline RL with Sparse Policies
by: Zhu, Lingwei, et al.
Published: (2025)

WS-GRPO: Weakly-Supervised Group-Relative Policy Optimization for Rollout-Efficient Reasoning
by: Mundada, Gagan, et al.
Published: (2026)

Your Offline Policy is Not Trustworthy: Bilevel Reinforcement Learning for Sequential Portfolio Optimization
by: Yuan, Haochen, et al.
Published: (2025)

DyDiff: Long-Horizon Rollout via Dynamics Diffusion for Offline Reinforcement Learning
by: Zhao, Hanye, et al.
Published: (2024)

Off-Policy Safe Reinforcement Learning with Constrained Optimistic Exploration
by: Li, Guopeng, et al.
Published: (2026)

BREAD: Branched Rollouts from Expert Anchors Bridge SFT & RL for Reasoning
by: Zhang, Xuechen, et al.
Published: (2025)

Optimistic Dual Averaging Unifies Modern Optimizers
by: Pethick, Thomas, et al.
Published: (2026)

DiffPoGAN: Diffusion Policies with Generative Adversarial Networks for Offline Reinforcement Learning
by: Hu, Xuemin, et al.
Published: (2024)

Inference Time Policy Optimization for Offline RL with Differentiable World Models
by: Deb, Rohan, et al.
Published: (2026)