Self-learning Monte Carlo with equivariant Transformer
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917834022453248 |
|---|---|
| author | Nagai, Yuki Tomiya, Akio |
| author_facet | Nagai, Yuki Tomiya, Akio |
| contents | Machine learning and deep learning have revolutionized computational physics, particularly the simulation of complex systems. Equivariance is essential for simulating physical systems because it imposes a strong inductive bias on the probability distribution described by a machine learning model. However, imposing symmetry on the model can sometimes lead to poor acceptance rates in self-learning Monte Carlo (SLMC). Here, we introduce a symmetry equivariant attention mechanism for SLMC, which can be systematically improved. We evaluate our architecture on a spin-fermion model (\textit{i.e.}, double exchange model) on a two-dimensional lattice. Our results show that the proposed method overcomes the poor acceptance rates of linear models and exhibits a similar scaling law to large language models, with model quality monotonically increasing with the number of layers. Our work paves the way for the development of more accurate and efficient Monte Carlo algorithms with machine learning for simulating complex physical systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2306_11527 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Self-learning Monte Carlo with equivariant Transformer Nagai, Yuki Tomiya, Akio Strongly Correlated Electrons Disordered Systems and Neural Networks High Energy Physics - Lattice Machine learning and deep learning have revolutionized computational physics, particularly the simulation of complex systems. Equivariance is essential for simulating physical systems because it imposes a strong inductive bias on the probability distribution described by a machine learning model. However, imposing symmetry on the model can sometimes lead to poor acceptance rates in self-learning Monte Carlo (SLMC). Here, we introduce a symmetry equivariant attention mechanism for SLMC, which can be systematically improved. We evaluate our architecture on a spin-fermion model (\textit{i.e.}, double exchange model) on a two-dimensional lattice. Our results show that the proposed method overcomes the poor acceptance rates of linear models and exhibits a similar scaling law to large language models, with model quality monotonically increasing with the number of layers. Our work paves the way for the development of more accurate and efficient Monte Carlo algorithms with machine learning for simulating complex physical systems. |
| title | Self-learning Monte Carlo with equivariant Transformer |
| topic | Strongly Correlated Electrons Disordered Systems and Neural Networks High Energy Physics - Lattice |
| url | https://arxiv.org/abs/2306.11527 |