Training-Driven Representational Geometry Modularization Predicts Brain Alignment in Language Models
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917257030926336 |
|---|---|
| author | Liu, Yixuan Ma, Zhiyuan Tang, Likai Gan, Runmin Zhang, Xinche Li, Jinhao Xie, Chao Song, Sen |
| author_facet | Liu, Yixuan Ma, Zhiyuan Tang, Likai Gan, Runmin Zhang, Xinche Li, Jinhao Xie, Chao Song, Sen |
| contents | How large language models (LLMs) align with the neural representation and computation of human language is a central question in cognitive science. Using representational geometry as a mechanistic lens, we addressed this by tracking entropy, curvature, and fMRI encoding scores throughout Pythia (70M-1B) training. We identified a geometric modularization where layers self-organize into stable low- and high-complexity clusters. The low-complexity module, characterized by reduced entropy and curvature, consistently better predicted human language network activity. This alignment followed heterogeneous spatial-temporal trajectories: rapid and stable in temporal regions (AntTemp, PostTemp), but delayed and dynamic in frontal areas (IFG, IFGorb). Crucially, reduced curvature remained a robust predictor of model-brain alignment even after controlling for training progress, an effect that strengthened with model scale. These results links training-driven geometric reorganization to temporal-frontal functional specialization, suggesting that representational smoothing facilitates neural-like linguistic processing. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_07539 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Training-Driven Representational Geometry Modularization Predicts Brain Alignment in Language Models Liu, Yixuan Ma, Zhiyuan Tang, Likai Gan, Runmin Zhang, Xinche Li, Jinhao Xie, Chao Song, Sen Neurons and Cognition Computation and Language How large language models (LLMs) align with the neural representation and computation of human language is a central question in cognitive science. Using representational geometry as a mechanistic lens, we addressed this by tracking entropy, curvature, and fMRI encoding scores throughout Pythia (70M-1B) training. We identified a geometric modularization where layers self-organize into stable low- and high-complexity clusters. The low-complexity module, characterized by reduced entropy and curvature, consistently better predicted human language network activity. This alignment followed heterogeneous spatial-temporal trajectories: rapid and stable in temporal regions (AntTemp, PostTemp), but delayed and dynamic in frontal areas (IFG, IFGorb). Crucially, reduced curvature remained a robust predictor of model-brain alignment even after controlling for training progress, an effect that strengthened with model scale. These results links training-driven geometric reorganization to temporal-frontal functional specialization, suggesting that representational smoothing facilitates neural-like linguistic processing. |
| title | Training-Driven Representational Geometry Modularization Predicts Brain Alignment in Language Models |
| topic | Neurons and Cognition Computation and Language |
| url | https://arxiv.org/abs/2602.07539 |