Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909018106101760 |
|---|---|
| author | Wurgaft, Daniel Rager, Can Kowal, Matthew Shyam, Vasudev Feucht, Sheridan Bhalla, Usha Haklay, Tal Bigelow, Eric Sarfati, Raphael McGrath, Thomas Lewis, Owen Merullo, Jack Goodman, Noah Fel, Thomas Geiger, Atticus Lubana, Ekdeep Singh |
| author_facet | Wurgaft, Daniel Rager, Can Kowal, Matthew Shyam, Vasudev Feucht, Sheridan Bhalla, Usha Haklay, Tal Bigelow, Eric Sarfati, Raphael McGrath, Thomas Lewis, Owen Merullo, Jack Goodman, Noah Fel, Thomas Geiger, Atticus Lubana, Ekdeep Singh |
| contents | Neural representations carry rich geometric structure; but does that structure causally shape behavior? To address this question, we intervene along paths through activation space defined by different geometries, and measure the behavioral trajectories they induce. In particular, we test whether interventions that respect the geometry of activation space will yield behaviors close to those the model exhibits naturally. Concretely, we first fit an activation manifold $M_h$ to representations and a behavior manifold $M_y$ to output probability distributions. We then test the link $M_h \leftrightarrow M_y$ via interventions: we find that steering along $M_h$, which we term manifold steering, yields behavioral trajectories that follow $M_y$, while linear steering -- which assumes a Euclidean geometry -- cuts through off-manifold regions and hence produces unnatural outputs. Moreover, optimizing interventions in activation space to produce paths along $M_y$ recovers activation trajectories that trace the curvature of $M_h$. We demonstrate this bidirectional relationship between the geometry of representation and behavior across tasks and modalities. In language models, we use reasoning tasks with cyclic and sequential geometries as well as in-context learning tasks with more complex graph geometries. In a video world model, we use a task with geometry corresponding to physical dynamics. Overall, our work shows that geometry in neural representation is not merely incidental, but is in fact the proper object for enabling principled control via intervention on internals. This recasts the core problem of steering from finding the right direction to finding the right geometry. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_05115 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior Wurgaft, Daniel Rager, Can Kowal, Matthew Shyam, Vasudev Feucht, Sheridan Bhalla, Usha Haklay, Tal Bigelow, Eric Sarfati, Raphael McGrath, Thomas Lewis, Owen Merullo, Jack Goodman, Noah Fel, Thomas Geiger, Atticus Lubana, Ekdeep Singh Machine Learning Neural representations carry rich geometric structure; but does that structure causally shape behavior? To address this question, we intervene along paths through activation space defined by different geometries, and measure the behavioral trajectories they induce. In particular, we test whether interventions that respect the geometry of activation space will yield behaviors close to those the model exhibits naturally. Concretely, we first fit an activation manifold $M_h$ to representations and a behavior manifold $M_y$ to output probability distributions. We then test the link $M_h \leftrightarrow M_y$ via interventions: we find that steering along $M_h$, which we term manifold steering, yields behavioral trajectories that follow $M_y$, while linear steering -- which assumes a Euclidean geometry -- cuts through off-manifold regions and hence produces unnatural outputs. Moreover, optimizing interventions in activation space to produce paths along $M_y$ recovers activation trajectories that trace the curvature of $M_h$. We demonstrate this bidirectional relationship between the geometry of representation and behavior across tasks and modalities. In language models, we use reasoning tasks with cyclic and sequential geometries as well as in-context learning tasks with more complex graph geometries. In a video world model, we use a task with geometry corresponding to physical dynamics. Overall, our work shows that geometry in neural representation is not merely incidental, but is in fact the proper object for enabling principled control via intervention on internals. This recasts the core problem of steering from finding the right direction to finding the right geometry. |
| title | Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2605.05115 |