Controlling the low-temperature Ising model using spatiotemporal Markov decision theory
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916554836279296 |
|---|---|
| author | de Jongh, M. C. Boucherie, Richard J. van Lieshout, M. N. M. |
| author_facet | de Jongh, M. C. Boucherie, Richard J. van Lieshout, M. N. M. |
| contents | We introduce the spatiotemporal Markov decision process (STMDP), a special type of Markov decision process that models sequential decision-making problems which are not only characterized by temporal, but also by spatial interaction structures. To illustrate the framework, we construct an STMDP inspired by the low-temperature two-dimensional Ising model on a finite, square lattice, evolving according to the Metropolis dynamics. We consider the situation in which an external decision maker aims to drive the system towards the all-plus configuration by flipping spins at specified moments in time. In order to analyze this problem, we construct an auxiliary MDP by means of a reduction of the configuration space to the local minima of the Hamiltonian. Leveraging the convenient form of this auxiliary MDP, we uncover the structure of the optimal policy by solving the Bellman equations in a recursive manner. Finally, we conduct a numerical study on the performance of the optimal policy obtained from the auxiliary MDP in the original Ising STMDP. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_03668 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Controlling the low-temperature Ising model using spatiotemporal Markov decision theory de Jongh, M. C. Boucherie, Richard J. van Lieshout, M. N. M. Optimization and Control Mathematical Physics Probability 90C40, 82B20 (Primary), 60G60 (Secondary) We introduce the spatiotemporal Markov decision process (STMDP), a special type of Markov decision process that models sequential decision-making problems which are not only characterized by temporal, but also by spatial interaction structures. To illustrate the framework, we construct an STMDP inspired by the low-temperature two-dimensional Ising model on a finite, square lattice, evolving according to the Metropolis dynamics. We consider the situation in which an external decision maker aims to drive the system towards the all-plus configuration by flipping spins at specified moments in time. In order to analyze this problem, we construct an auxiliary MDP by means of a reduction of the configuration space to the local minima of the Hamiltonian. Leveraging the convenient form of this auxiliary MDP, we uncover the structure of the optimal policy by solving the Bellman equations in a recursive manner. Finally, we conduct a numerical study on the performance of the optimal policy obtained from the auxiliary MDP in the original Ising STMDP. |
| title | Controlling the low-temperature Ising model using spatiotemporal Markov decision theory |
| topic | Optimization and Control Mathematical Physics Probability 90C40, 82B20 (Primary), 60G60 (Secondary) |
| url | https://arxiv.org/abs/2501.03668 |