LLM Reasoning as Trajectories: Step-Specific Representation Geometry and Correctness Signals
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910107689811968 |
|---|---|
| author | Sun, Lihao Dong, Hang Qiao, Bo Lin, Qingwei Zhang, Dongmei Rajmohan, Saravan |
| author_facet | Sun, Lihao Dong, Hang Qiao, Bo Lin, Qingwei Zhang, Dongmei Rajmohan, Saravan |
| contents | This work characterizes large language models' chain-of-thought generation as a structured trajectory through representation space. We show that mathematical reasoning traverses functionally ordered, step-specific subspaces that become increasingly separable with layer depth. This structure already exists in base models, while reasoning training primarily accelerates convergence toward termination-related subspaces rather than introducing new representational organization. While early reasoning steps follow similar trajectories, correct and incorrect solutions diverge systematically at late stages. This late-stage divergence enables mid-reasoning prediction of final-answer correctness with ROC-AUC up to 0.87. Furthermore, we introduce trajectory-based steering, an inference-time intervention framework that enables reasoning correction and length control based on derived ideal trajectories. Together, these results establish reasoning trajectories as a geometric lens for interpreting, predicting, and controlling LLM reasoning behavior. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_05655 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | LLM Reasoning as Trajectories: Step-Specific Representation Geometry and Correctness Signals Sun, Lihao Dong, Hang Qiao, Bo Lin, Qingwei Zhang, Dongmei Rajmohan, Saravan Computation and Language Artificial Intelligence Machine Learning This work characterizes large language models' chain-of-thought generation as a structured trajectory through representation space. We show that mathematical reasoning traverses functionally ordered, step-specific subspaces that become increasingly separable with layer depth. This structure already exists in base models, while reasoning training primarily accelerates convergence toward termination-related subspaces rather than introducing new representational organization. While early reasoning steps follow similar trajectories, correct and incorrect solutions diverge systematically at late stages. This late-stage divergence enables mid-reasoning prediction of final-answer correctness with ROC-AUC up to 0.87. Furthermore, we introduce trajectory-based steering, an inference-time intervention framework that enables reasoning correction and length control based on derived ideal trajectories. Together, these results establish reasoning trajectories as a geometric lens for interpreting, predicting, and controlling LLM reasoning behavior. |
| title | LLM Reasoning as Trajectories: Step-Specific Representation Geometry and Correctness Signals |
| topic | Computation and Language Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2604.05655 |