A note on the dynamics of extended-context disordered kinetic spin models
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917149962928128 |
|---|---|
| author | Zavatone-Veth, Jacob A. Pehlevan, Cengiz |
| author_facet | Zavatone-Veth, Jacob A. Pehlevan, Cengiz |
| contents | Inspired by striking advances in language modeling, there has recently been much interest in developing autogressive sequence models that are amenable to analytical study. In this short note, we consider extensions of simple disordered kinetic glass models from statistical physics. These models have tunable correlations, are easy to sample, and can be solved exactly when the state space dimension is large. In particular, we give an expository derivation of the dynamical mean field theories that describe their asymptotic statistics. We therefore propose that they constitute an interesting set of toy models for autoregressive sequence generation, in which one might study learning dynamics. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_18461 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A note on the dynamics of extended-context disordered kinetic spin models Zavatone-Veth, Jacob A. Pehlevan, Cengiz Disordered Systems and Neural Networks Inspired by striking advances in language modeling, there has recently been much interest in developing autogressive sequence models that are amenable to analytical study. In this short note, we consider extensions of simple disordered kinetic glass models from statistical physics. These models have tunable correlations, are easy to sample, and can be solved exactly when the state space dimension is large. In particular, we give an expository derivation of the dynamical mean field theories that describe their asymptotic statistics. We therefore propose that they constitute an interesting set of toy models for autoregressive sequence generation, in which one might study learning dynamics. |
| title | A note on the dynamics of extended-context disordered kinetic spin models |
| topic | Disordered Systems and Neural Networks |
| url | https://arxiv.org/abs/2507.18461 |