Are language models aware of the road not taken? Token-level uncertainty and hidden state dynamics
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914141092970496 |
|---|---|
| author | Zur, Amir Geiger, Atticus Lubana, Ekdeep Singh Bigelow, Eric |
| author_facet | Zur, Amir Geiger, Atticus Lubana, Ekdeep Singh Bigelow, Eric |
| contents | When a language model generates text, the selection of individual tokens might lead it down very different reasoning paths, making uncertainty difficult to quantify. In this work, we consider whether reasoning language models represent the alternate paths that they could take during generation. To test this hypothesis, we use hidden activations to control and predict a language model's uncertainty during chain-of-thought reasoning. In our experiments, we find a clear correlation between how uncertain a model is at different tokens, and how easily the model can be steered by controlling its activations. This suggests that activation interventions are most effective when there are alternate paths available to the model -- in other words, when it has not yet committed to a particular final answer. We also find that hidden activations can predict a model's future outcome distribution, demonstrating that models implicitly represent the space of possible paths. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_04527 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Are language models aware of the road not taken? Token-level uncertainty and hidden state dynamics Zur, Amir Geiger, Atticus Lubana, Ekdeep Singh Bigelow, Eric Computation and Language Artificial Intelligence When a language model generates text, the selection of individual tokens might lead it down very different reasoning paths, making uncertainty difficult to quantify. In this work, we consider whether reasoning language models represent the alternate paths that they could take during generation. To test this hypothesis, we use hidden activations to control and predict a language model's uncertainty during chain-of-thought reasoning. In our experiments, we find a clear correlation between how uncertain a model is at different tokens, and how easily the model can be steered by controlling its activations. This suggests that activation interventions are most effective when there are alternate paths available to the model -- in other words, when it has not yet committed to a particular final answer. We also find that hidden activations can predict a model's future outcome distribution, demonstrating that models implicitly represent the space of possible paths. |
| title | Are language models aware of the road not taken? Token-level uncertainty and hidden state dynamics |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2511.04527 |