Cover Image

Scaling Test-Time Compute Without Verification or RL is Suboptimal

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Setlur, Amrith, Rajaraman, Nived, Levine, Sergey, Kumar, Aviral
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

RL on Incorrect Synthetic Data Scales the Efficiency of LLM Math Reasoning by Eight-Fold
by: Setlur, Amrith, et al.
Published: (2024)

e3: Learning to Explore Enables Extrapolation of Test-Time Compute for LLMs
by: Setlur, Amrith, et al.
Published: (2025)

Optimizing Test-Time Compute via Meta Reinforcement Fine-Tuning
by: Qu, Yuxiao, et al.
Published: (2025)

POPE: Learning to Reason on Hard Problems via Privileged On-Policy Exploration
by: Qu, Yuxiao, et al.
Published: (2026)

Reasoning Cache: Continual Improvement Over Long Horizons via Short-Horizon RL
by: Wu, Ian, et al.
Published: (2026)

Reuse your FLOPs: Scaling RL on Hard Problems by Conditioning on Very Off-Policy Prefixes
by: Setlur, Amrith, et al.
Published: (2026)

Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning
by: Setlur, Amrith, et al.
Published: (2024)

Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
by: Snell, Charlie, et al.
Published: (2024)

RLAD: Training LLMs to Discover Abstractions for Solving Reasoning Problems
by: Qu, Yuxiao, et al.
Published: (2025)

InT: Self-Proposed Interventions Enable Credit Assignment in LLM Reasoning
by: Yang, Matthew Y. R., et al.
Published: (2026)

ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL
by: Zhou, Yifei, et al.
Published: (2024)

Toward a Theory of Tokenization in LLMs
by: Rajaraman, Nived, et al.
Published: (2024)

$\texttt{SPECS}$: Faster Test-Time Scaling through Speculative Drafts
by: Cemri, Mert, et al.
Published: (2025)

What Do Learning Dynamics Reveal About Generalization in LLM Reasoning?
by: Kang, Katie, et al.
Published: (2024)

QED-Nano: Teaching a Tiny Model to Prove Hard Theorems
by: LM-Provers, et al.
Published: (2026)

Deep Neural Networks Tend To Extrapolate Predictably
by: Kang, Katie, et al.
Published: (2023)

RLAC: Reinforcement Learning with Adversarial Critic for Free-Form Generation Tasks
by: Wu, Mian, et al.
Published: (2025)

Unfamiliar Finetuning Examples Control How Language Models Hallucinate
by: Kang, Katie, et al.
Published: (2024)

Compute-Optimal Scaling for Value-Based Deep RL
by: Fu, Preston, et al.
Published: (2025)

Zero-Overhead Introspection for Adaptive Test-Time Compute
by: Manvi, Rohin, et al.
Published: (2025)

Transformers on Markov Data: Constant Depth Suffices
by: Rajaraman, Nived, et al.
Published: (2024)

IsoCompute Playbook: Optimally Scaling Sampling Compute for LLM RL
by: Cheng, Zhoujun, et al.
Published: (2026)

Thinking vs. Doing: Agents that Reason by Scaling Test-Time Interaction
by: Shen, Junhong, et al.
Published: (2025)

Is Value Learning Really the Main Bottleneck in Offline RL?
by: Park, Seohong, et al.
Published: (2024)

Value-Based Deep RL Scales Predictably
by: Rybkin, Oleh, et al.
Published: (2025)

Test-Time Scaling Makes Overtraining Compute-Optimal
by: Roberts, Nicholas, et al.
Published: (2026)

Computational Intractability of Strategizing against Online Learners
by: Assos, Angelos, et al.
Published: (2025)

Interactive Learning of Single-Index Models via Stochastic Gradient Descent
by: Rajaraman, Nived, et al.
Published: (2026)

Evaluating & Reducing Deceptive Dialogue From Language Models with Multi-turn RL
by: Abdulhai, Marwa, et al.
Published: (2025)

MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention
by: MiniMax, et al.
Published: (2025)

Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
by: Geiping, Jonas, et al.
Published: (2025)

ReVISE: Learning to Refine at Test-Time via Intrinsic Self-Verification
by: Lee, Hyunseok, et al.
Published: (2025)

Horizon Reduction Makes RL Scalable
by: Park, Seohong, et al.
Published: (2025)

Scaling Test-Time Compute for Agentic Coding
by: Kim, Joongwon, et al.
Published: (2026)

Test-Time Speculation
by: Kumar, Avinash, et al.
Published: (2026)

Budget-aware Test-time Scaling via Discriminative Verification
by: Montgomery, Kyle, et al.
Published: (2025)

Log-Augmented Generation: Scaling Test-Time Reasoning with Reusable Computation
by: Chen, Peter Baile, et al.
Published: (2025)

Strategic Scaling of Test-Time Compute: A Bandit Learning Approach
by: Zuo, Bowen, et al.
Published: (2025)

Provable Scaling Laws for the Test-Time Compute of Large Language Models
by: Chen, Yanxi, et al.
Published: (2024)

Kinetics: Rethinking Test-Time Scaling Laws
by: Sadhukhan, Ranajoy, et al.
Published: (2025)