Cover Image

Residual Q-Learning: Offline and Online Policy Customization without Value

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Chenran, Tang, Chen, Nishimura, Haruki, Mercat, Jean, Tomizuka, Masayoshi, Zhan, Wei
Format:	Preprint
Published:	2023
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MEReQ: Max-Ent Residual-Q Inverse RL for Sample-Efficient Alignment from Intervention
by: Chen, Yuxin, et al.
Published: (2024)

Skill-Critic: Refining Learned Skills for Hierarchical Reinforcement Learning
by: Hao, Ce, et al.
Published: (2023)

Residual-MPPI: Online Policy Customization for Continuous Control
by: Wang, Pengcheng, et al.
Published: (2024)

Grounded Relational Inference: Domain Knowledge Driven Explainable Autonomous Driving
by: Tang, Chen, et al.
Published: (2021)

Residual Policy Gradient: A Reward View of KL-regularized Objective
by: Wang, Pengcheng, et al.
Published: (2025)

Offline Reinforcement Learning with Generative Trajectory Policies
by: Feng, Xinsong, et al.
Published: (2025)

Mean Flow Policy with Instantaneous Velocity Constraint for One-step Action Generation
by: Zhan, Guojian, et al.
Published: (2026)

FlowQ: Energy-Guided Flow Policies for Offline Reinforcement Learning
by: Alles, Marvin, et al.
Published: (2025)

ENOTO: Improving Offline-to-Online Reinforcement Learning with Q-Ensembles
by: Zhao, Kai, et al.
Published: (2023)

Improving Offline-to-Online Reinforcement Learning with Q Conditioned State Entropy Exploration
by: Zhang, Ziqi, et al.
Published: (2023)

On the Complexity of Offline Reinforcement Learning with $Q^\star$-Approximation and Partial Coverage
by: Liu, Haolin, et al.
Published: (2026)

Online Optimization for Offline Safe Reinforcement Learning
by: Chemingui, Yassine, et al.
Published: (2025)

Diffusion Policies with Value-Conditional Optimization for Offline Reinforcement Learning
by: Ma, Yunchang, et al.
Published: (2025)

Safe Flow Q-Learning: Offline Safe Reinforcement Learning with Reachability-Based Flow Policies
by: Tayal, Mumuksh, et al.
Published: (2026)

Guided Flow Policy: Learning from High-Value Actions in Offline Reinforcement Learning
by: Tiofack, Franki Nguimatsia, et al.
Published: (2025)

COOPO: Cyclic Offline-Online Policy Optimization Algorithm
by: Liu, Qisai, et al.
Published: (2026)

Bootstrap Off-policy with World Model
by: Zhan, Guojian, et al.
Published: (2025)

Expert Q-learning: Deep Reinforcement Learning with Coarse State Values from Offline Expert Examples
by: Meng, Li, et al.
Published: (2021)

Driv3R: Learning Dense 4D Reconstruction for Autonomous Driving
by: Fei, Xin, et al.
Published: (2024)

In-Context Compositional Q-Learning for Offline Reinforcement Learning
by: Xu, Qiushui, et al.
Published: (2025)

Imagination-Limited Q-Learning for Offline Reinforcement Learning
by: Liu, Wenhui, et al.
Published: (2025)

Mildly Conservative Q-Learning for Offline Reinforcement Learning
by: Lyu, Jiafei, et al.
Published: (2022)

Belief-Based Offline Reinforcement Learning for Delay-Robust Policy Optimization
by: Zhan, Simon Sinong, et al.
Published: (2025)

Mind Your Entropy: From Maximum Entropy to Trajectory Entropy-Constrained RL
by: Zhan, Guojian, et al.
Published: (2025)

Maximizing Alignment with Minimal Feedback: Efficiently Learning Rewards for Visuomotor Robot Policy Alignment
by: Tian, Ran, et al.
Published: (2024)

When to Trust Your Simulator: Dynamics-Aware Hybrid Offline-and-Online Reinforcement Learning
by: Niu, Haoyi, et al.
Published: (2022)

Adaptive Neighborhood-Constrained Q Learning for Offline Reinforcement Learning
by: Mao, Yixiu, et al.
Published: (2025)

Exclusively Penalized Q-learning for Offline Reinforcement Learning
by: Yeom, Junghyuk, et al.
Published: (2024)

Adaptive Policy Selection and Fine-Tuning under Interaction Budgets for Offline-to-Online Reinforcement Learning
by: Bozkurt, Alper Kamil, et al.
Published: (2026)

VIPO: Value Function Inconsistency Penalized Offline Reinforcement Learning
by: Chen, Xuyang, et al.
Published: (2025)

Stable Offline Value Function Learning with Bisimulation-based Representations
by: Pavse, Brahma S., et al.
Published: (2024)

Preferred-Action-Optimized Diffusion Policies for Offline Reinforcement Learning
by: Zhang, Tianle, et al.
Published: (2024)

Action-Free Offline-to-Online RL via Discretised State Policies
by: Neggatu, Natinael Solomon, et al.
Published: (2026)

Value-Incentivized Preference Optimization: A Unified Approach to Online and Offline RLHF
by: Cen, Shicong, et al.
Published: (2024)

Efficient Online RL Fine Tuning with Offline Pre-trained Policy Only
by: Xiao, Wei, et al.
Published: (2025)

ACL-QL: Adaptive Conservative Level in Q-Learning for Offline Reinforcement Learning
by: Wu, Kun, et al.
Published: (2024)

Constraint-Adaptive Policy Switching for Offline Safe Reinforcement Learning
by: Chemingui, Yassine, et al.
Published: (2024)

RLSynC: Offline-Online Reinforcement Learning for Synthon Completion
by: Baker, Frazier N., et al.
Published: (2023)

The Three Regimes of Offline-to-Online Reinforcement Learning
by: Li, Lu, et al.
Published: (2025)

Model-based Offline Reinforcement Learning with Lower Expectile Q-Learning
by: Park, Kwanyoung, et al.
Published: (2024)