Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Nakamoto, Mitsuhiko, Zhai, Yuexiang, Singh, Anikait, Mark, Max Sobol, Ma, Yi, Finn, Chelsea, Kumar, Aviral, Levine, Sergey
Format:	Preprint
Published:	2023
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Policy Agnostic RL: Offline RL and Online RL Fine-Tuning of Any Class and Backbone
by: Mark, Max Sobol, et al.
Published: (2024)

Steering Your Generalists: Improving Robotic Foundation Models via Value Guidance
by: Nakamoto, Mitsuhiko, et al.
Published: (2024)

Efficient Online Reinforcement Learning Fine-Tuning Need Not Retain Offline Data
by: Zhou, Zhiyuan, et al.
Published: (2024)

Is Value Learning Really the Main Bottleneck in Offline RL?
by: Park, Seohong, et al.
Published: (2024)

Preference Fine-Tuning of LLMs Should Leverage Suboptimal, On-Policy Data
by: Tajwar, Fahim, et al.
Published: (2024)

RLAD: Training LLMs to Discover Abstractions for Solving Reasoning Problems
by: Qu, Yuxiao, et al.
Published: (2025)

D5RL: Diverse Datasets for Data-Driven Deep Reinforcement Learning
by: Rafailov, Rafael, et al.
Published: (2024)

Efficient Online RL Fine Tuning with Offline Pre-trained Policy Only
by: Xiao, Wei, et al.
Published: (2025)

ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL
by: Zhou, Yifei, et al.
Published: (2024)

MOTO: Offline Pre-training to Online Fine-tuning for Model-based Robot Learning
by: Rafailov, Rafael, et al.
Published: (2024)

Scaling Test-Time Compute Without Verification or RL is Suboptimal
by: Setlur, Amrith, et al.
Published: (2025)

Posterior Behavioral Cloning: Pretraining BC Policies for Efficient RL Finetuning
by: Wagenmaker, Andrew, et al.
Published: (2025)

DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning
by: Bai, Hao, et al.
Published: (2024)

Horizon Reduction Makes RL Scalable
by: Park, Seohong, et al.
Published: (2025)

OGBench: Benchmarking Offline Goal-Conditioned RL
by: Park, Seohong, et al.
Published: (2024)

Optimistic Critic Reconstruction and Constrained Fine-Tuning for General Offline-to-Online RL
by: Luo, Qin-Wen, et al.
Published: (2024)

Scalable Offline Model-Based RL with Action Chunks
by: Park, Kwanyoung, et al.
Published: (2025)

ViVa: Video-Trained Value Functions for Guiding Online RL from Diverse Data
by: Dashora, Nitish, et al.
Published: (2025)

Planning without Search: Refining Frontier LLMs with Offline Goal-Conditioned RL
by: Hong, Joey, et al.
Published: (2025)

Value-Based Deep RL Scales Predictably
by: Rybkin, Oleh, et al.
Published: (2025)

HIQL: Offline Goal-Conditioned RL with Latent States as Actions
by: Park, Seohong, et al.
Published: (2023)

Compute-Optimal Scaling for Value-Based Deep RL
by: Fu, Preston, et al.
Published: (2025)

Integrating Offline Pre-Training with Online Fine-Tuning: A Reinforcement Learning Approach for Robot Social Navigation
by: Su, Run, et al.
Published: (2025)

RLIF: Interactive Imitation Learning as Reinforcement Learning
by: Luo, Jianlan, et al.
Published: (2023)

Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success
by: Kim, Moo Jin, et al.
Published: (2025)

Digi-Q: Learning Q-Value Functions for Training Device-Control Agents
by: Bai, Hao, et al.
Published: (2025)

Offline RL by Reward-Weighted Fine-Tuning for Conversation Optimization
by: Mukherjee, Subhojyoti, et al.
Published: (2025)

Vision-Language Models Provide Promptable Representations for Reinforcement Learning
by: Chen, William, et al.
Published: (2024)

Test-Time Alignment via Hypothesis Reweighting
by: Lee, Yoonho, et al.
Published: (2024)

BPP: Long-Context Robot Imitation Learning by Focusing on Key History Frames
by: Mark, Max Sobol, et al.
Published: (2026)

Online Pre-Training for Offline-to-Online Reinforcement Learning
by: Shin, Yongjae, et al.
Published: (2025)

Steering Your Diffusion Policy with Latent Space Reinforcement Learning
by: Wagenmaker, Andrew, et al.
Published: (2025)

Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning
by: Zhai, Yuexiang, et al.
Published: (2024)

Visual Pre-Training on Unlabeled Images using Reinforcement Learning
by: Ghosh, Dibya, et al.
Published: (2025)

Is Offline Decision Making Possible with Only Few Samples? Reliable Decisions in Data-Starved Bandits via Trust Region Enhancement
by: Zhang, Ruiqi, et al.
Published: (2024)

Prompt-Tuning Bandits: Enabling Few-Shot Generalization for Efficient Multi-Task Offline RL
by: Rietz, Finn, et al.
Published: (2025)

SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training
by: Chu, Tianzhe, et al.
Published: (2025)

Reflect-RL: Two-Player Online RL Fine-Tuning for LMs
by: Zhou, Runlong, et al.
Published: (2024)

floq: Training Critics via Flow-Matching for Scaling Compute in Value-Based RL
by: Agrawalla, Bhavya, et al.
Published: (2025)

FSPO: Few-Shot Optimization of Synthetic Preferences Personalizes to Real Users
by: Singh, Anikait, et al.
Published: (2025)