Cover Image

Stop Regressing: Training Value Functions via Classification for Scalable Deep RL

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Farebrother, Jesse, Orbay, Jordi, Vuong, Quan, Taïga, Adrien Ali, Chebotar, Yevgen, Xiao, Ted, Irpan, Alex, Levine, Sergey, Castro, Pablo Samuel, Faust, Aleksandra, Kumar, Aviral, Agarwal, Rishabh
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Is Value Learning Really the Main Bottleneck in Offline RL?
by: Park, Seohong, et al.
Published: (2024)

Horizon Reduction Makes RL Scalable
by: Park, Seohong, et al.
Published: (2025)

Value-Based Deep RL Scales Predictably
by: Rybkin, Oleh, et al.
Published: (2025)

RT-H: Action Hierarchies Using Language
by: Belkhale, Suneel, et al.
Published: (2024)

Compute-Optimal Scaling for Value-Based Deep RL
by: Fu, Preston, et al.
Published: (2025)

Scaling Test-Time Compute Without Verification or RL is Suboptimal
by: Setlur, Amrith, et al.
Published: (2025)

Steering Your Generalists: Improving Robotic Foundation Models via Value Guidance
by: Nakamoto, Mitsuhiko, et al.
Published: (2024)

ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL
by: Zhou, Yifei, et al.
Published: (2024)

METRA: Scalable Unsupervised RL with Metric-Aware Abstraction
by: Park, Seohong, et al.
Published: (2023)

CALE: Continuous Arcade Learning Environment
by: Farebrother, Jesse, et al.
Published: (2024)

Scalable Offline Model-Based RL with Action Chunks
by: Park, Kwanyoung, et al.
Published: (2025)

Transitive RL: Value Learning via Divide and Conquer
by: Park, Seohong, et al.
Published: (2025)

Inference-Aware Fine-Tuning for Best-of-N Sampling in Large Language Models
by: Chow, Yinlam, et al.
Published: (2024)

What does Hirsch index evolution explain us? A case study: Turkish Journal of Chemistry
by: Metin Orbay
Published: (2007)

Digi-Q: Learning Q-Value Functions for Training Device-Control Agents
by: Bai, Hao, et al.
Published: (2025)

ViVa: Video-Trained Value Functions for Guiding Online RL from Diverse Data
by: Dashora, Nitish, et al.
Published: (2025)

DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning
by: Bai, Hao, et al.
Published: (2024)

Consistency Training Helps Stop Sycophancy and Jailbreaks
by: Irpan, Alex, et al.
Published: (2025)

Vision-Language Models Provide Promptable Representations for Reinforcement Learning
by: Chen, William, et al.
Published: (2024)

Putting the Value Back in RL: Better Test-Time Scaling by Unifying LLM Reasoners With Verifiers
by: Sareen, Kusha, et al.
Published: (2025)

Foundations of Multivariate Distributional Reinforcement Learning
by: Wiltzer, Harley, et al.
Published: (2024)

RACER: Epistemic Risk-Sensitive RL Enables Fast Driving with Fewer Crashes
by: Stachowicz, Kyle, et al.
Published: (2024)

floq: Training Critics via Flow-Matching for Scaling Compute in Value-Based RL
by: Agrawalla, Bhavya, et al.
Published: (2025)

Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuning
by: Nakamoto, Mitsuhiko, et al.
Published: (2023)

Mixtures of Experts Unlock Parameter Scaling for Deep RL
by: Obando-Ceron, Johan, et al.
Published: (2024)

Efficient Online Reinforcement Learning Fine-Tuning Need Not Retain Offline Data
by: Zhou, Zhiyuan, et al.
Published: (2024)

Unfamiliar Finetuning Examples Control How Language Models Hallucinate
by: Kang, Katie, et al.
Published: (2024)

RLAC: Reinforcement Learning with Adversarial Critic for Free-Form Generation Tasks
by: Wu, Mian, et al.
Published: (2025)

Generative Verifiers: Reward Modeling as Next-Token Prediction
by: Zhang, Lunjun, et al.
Published: (2024)

Planning without Search: Refining Frontier LLMs with Offline Goal-Conditioned RL
by: Hong, Joey, et al.
Published: (2025)

OGBench: Benchmarking Offline Goal-Conditioned RL
by: Park, Seohong, et al.
Published: (2024)

HIQL: Offline Goal-Conditioned RL with Latent States as Actions
by: Park, Seohong, et al.
Published: (2023)

D5RL: Diverse Datasets for Data-Driven Deep Reinforcement Learning
by: Rafailov, Rafael, et al.
Published: (2024)

Early Stopping for Regression Trees
by: Miftachov, Ratmir, et al.
Published: (2025)

Temporal Difference Flows
by: Farebrother, Jesse, et al.
Published: (2025)

Analysis of modern systems for electronic control of knowledge of future specialists of pharmaceutical profile based on moodle: problems and prospects of improvement
by: Yevgen Diiev
Published: (2022)

Sub-Cubic Quantum Gate Synthesis via Stochastic Commutator Decomposition
by: Kotukh, Yevgen
Published: (2026)

Estudio comparativo de los métodos del cálculo de la transferencia de calor en bancos de tubos
by: Yevgen Pysmennyy
Published: (2010)

Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning
by: Setlur, Amrith, et al.
Published: (2024)

Asynchronous RLHF: Faster and More Efficient Off-Policy RL for Language Models
by: Noukhovitch, Michael, et al.
Published: (2024)