Inference-Time Rethinking with Latent Thought Vectors for Math Reasoning
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912884431257600 |
|---|---|
| author | Kong, Deqian Zhao, Minglu Qin, Aoyang Pang, Bo Tao, Chenxin Hartmann, David Honig, Edouardo Xu, Dehong Kumar, Amit Sarte, Matt Li, Chuan Xie, Jianwen Wu, Ying Nian |
| author_facet | Kong, Deqian Zhao, Minglu Qin, Aoyang Pang, Bo Tao, Chenxin Hartmann, David Honig, Edouardo Xu, Dehong Kumar, Amit Sarte, Matt Li, Chuan Xie, Jianwen Wu, Ying Nian |
| contents | Standard chain-of-thought reasoning generates a solution in a single forward pass, committing irrevocably to each token and lacking a mechanism to recover from early errors. We introduce Inference-Time Rethinking, a generative framework that enables iterative self-correction by decoupling declarative latent thought vectors from procedural generation. We factorize reasoning into a continuous latent thought vector (what to reason about) and a decoder that verbalizes the trace conditioned on this vector (how to reason). Beyond serving as a declarative buffer, latent thought vectors compress the reasoning structure into a continuous representation that abstracts away surface-level token variability, making gradient-based optimization over reasoning strategies well-posed. Our prior model maps unstructured noise to a learned manifold of valid reasoning patterns, and at test time we employ a Gibbs-style procedure that alternates between generating a candidate trace and optimizing the latent vector to better explain that trace, effectively navigating the latent manifold to refine the reasoning strategy. Training a 0.2B-parameter model from scratch on GSM8K, our method with 30 rethinking iterations surpasses baselines with 10 to 15 times more parameters, including a 3B counterpart. This result demonstrates that effective mathematical reasoning can emerge from sophisticated inference-time computation rather than solely from massive parameter counts. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_06584 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Inference-Time Rethinking with Latent Thought Vectors for Math Reasoning Kong, Deqian Zhao, Minglu Qin, Aoyang Pang, Bo Tao, Chenxin Hartmann, David Honig, Edouardo Xu, Dehong Kumar, Amit Sarte, Matt Li, Chuan Xie, Jianwen Wu, Ying Nian Computation and Language Machine Learning Standard chain-of-thought reasoning generates a solution in a single forward pass, committing irrevocably to each token and lacking a mechanism to recover from early errors. We introduce Inference-Time Rethinking, a generative framework that enables iterative self-correction by decoupling declarative latent thought vectors from procedural generation. We factorize reasoning into a continuous latent thought vector (what to reason about) and a decoder that verbalizes the trace conditioned on this vector (how to reason). Beyond serving as a declarative buffer, latent thought vectors compress the reasoning structure into a continuous representation that abstracts away surface-level token variability, making gradient-based optimization over reasoning strategies well-posed. Our prior model maps unstructured noise to a learned manifold of valid reasoning patterns, and at test time we employ a Gibbs-style procedure that alternates between generating a candidate trace and optimizing the latent vector to better explain that trace, effectively navigating the latent manifold to refine the reasoning strategy. Training a 0.2B-parameter model from scratch on GSM8K, our method with 30 rethinking iterations surpasses baselines with 10 to 15 times more parameters, including a 3B counterpart. This result demonstrates that effective mathematical reasoning can emerge from sophisticated inference-time computation rather than solely from massive parameter counts. |
| title | Inference-Time Rethinking with Latent Thought Vectors for Math Reasoning |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2602.06584 |