To Backtrack or Not to Backtrack: When Sequential Search Limits Model Reasoning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Qin, Tian, Alvarez-Melis, David, Jelassi, Samy, Malach, Eran
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Repeat After Me: Transformers are Better than State Space Models at Copying
by: Jelassi, Samy, et al.
Published: (2024)

Universal Length Generalization with Turing Programs
by: Hou, Kaiying, et al.
Published: (2024)

Adaptive Backtracking Line Search
by: Cavalcanti, Joao V., et al.
Published: (2024)

The Role of Sparsity for Length Generalization in Transformers
by: Golowich, Noah, et al.
Published: (2025)

Let Me Think! A Long Chain-of-Thought Can Be Worth Exponentially Many Short Ones
by: Mirtaheri, Parsa, et al.
Published: (2025)

LoRA Soups: Merging LoRAs for Practical Skill Composition Tasks
by: Prabhakar, Akshara, et al.
Published: (2024)

Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretraining
by: Zhao, Rosie, et al.
Published: (2025)

Mixture of Parrots: Experts improve memorization more than reasoning
by: Jelassi, Samy, et al.
Published: (2024)

The Power of Random Features and the Limits of Distribution-Free Gradient Descent
by: Karchmer, Ari, et al.
Published: (2025)

Can Transformers Learn to Verify During Backtracking Search?
by: Phua, Yin Jun, et al.
Published: (2026)

Backtracking Improves Generation Safety
by: Zhang, Yiming, et al.
Published: (2024)

Reinforcement Learning with Backtracking Feedback
by: Sel, Bilgehan, et al.
Published: (2026)

Auto-Regressive Next-Token Predictors are Universal Learners
by: Malach, Eran
Published: (2023)

Adaptive Backtracking for Privacy Protection in Large Language Models
by: Yao, Zhihao, et al.
Published: (2025)

Deep Backtracking Counterfactuals for Causally Compliant Explanations
by: Kladny, Klaus-Rudolf, et al.
Published: (2023)

Natural Counterfactuals With Necessary Backtracking
by: Hao, Guang-Yuan, et al.
Published: (2024)

SequenceMatch: Imitation Learning for Autoregressive Sequence Modelling with Backtracking
by: Cundy, Chris, et al.
Published: (2023)

Collective Model Intelligence Requires Compatible Specialization
by: Pari, Jyothish, et al.
Published: (2024)

Beyond the Frontier: Stochastic Backtracking for Efficient Test-Time Scaling
by: Tran, Dao, et al.
Published: (2026)

Taming Imperfect Process Verifiers: A Sampling Perspective on Backtracking
by: Rohatgi, Dhruv, et al.
Published: (2025)

Decomposing Elements of Problem Solving: What "Math" Does RL Teach?
by: Qin, Tian, et al.
Published: (2025)

Emergent Search and Backtracking in Latent Reasoning Models
by: Cui, Jasmine, et al.
Published: (2026)

One-Step Forward and Backtrack: Overcoming Zig-Zagging in Loss-Aware Quantization Training
by: Ma, Lianbo, et al.
Published: (2024)

Dynamic Backtracking in GFlowNets: Enhancing Decision Steps with Reward-Dependent Adjustment Mechanisms
by: Guo, Shuai, et al.
Published: (2024)

IterGen: Iterative Semantic-aware Structured LLM Generation with Backtracking
by: Ugare, Shubham, et al.
Published: (2024)

How Does Overparameterization Affect Features?
by: Duzgun, Ahmet Cagri, et al.
Published: (2024)

Cardiac Evidence Backtracking for Eating Behavior Monitoring using Collocative Electrocardiogram Imagining
by: Zhang, Xu-Lu, et al.
Published: (2025)

A New Approach to Backtracking Counterfactual Explanations: A Unified Causal Framework for Efficient Model Interpretability
by: Fatemi, Pouria, et al.
Published: (2025)

Saber: An Efficient Sampling with Adaptive Acceleration and Backtracking Enhanced Remasking for Diffusion Language Model
by: Dong, Yihong, et al.
Published: (2025)

Distributional Dataset Distillation with Subtask Decomposition
by: Qin, Tian, et al.
Published: (2024)

Q-Probe: A Lightweight Approach to Reward Maximization for Language Models
by: Li, Kenneth, et al.
Published: (2024)

LLM Priors for ERM over Programs
by: Singhal, Shivam, et al.
Published: (2025)

How Reinforcement Learning After Next-Token Prediction Facilitates Learning
by: Tsilivis, Nikolaos, et al.
Published: (2025)

Sometimes I am a Tree: Data Drives Unstable Hierarchical Generalization
by: Qin, Tian, et al.
Published: (2024)

Backtracking for Safety
by: Sel, Bilgehan, et al.
Published: (2025)

Backtrackable Inprocessing
by: Nadel, Alexander
Published: (2026)

A Label is Worth a Thousand Images in Dataset Distillation
by: Qin, Tian, et al.
Published: (2024)

Backtracking New Q-Newton's method, Newton's flow, Voronoi's diagram and Stochastic root finding
by: Fornaess, John Erik, et al.
Published: (2024)

Unravelling Causal Genetic Biomarkers of Alzheimer's Disease via Neuron to Gene-token Backtracking in Neural Architecture: A Groundbreaking Reverse-Gene-Finder Approach
by: Li, Victor OK, et al.
Published: (2025)

BacktrackAgent: Enhancing GUI Agent with Error Detection and Backtracking Mechanism
by: Wu, Qinzhuo, et al.
Published: (2025)