Crystalformer: Infinitely Connected Attention for Periodic Structure Encoding
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909139779715072 |
|---|---|
| author | Taniai, Tatsunori Igarashi, Ryo Suzuki, Yuta Chiba, Naoya Saito, Kotaro Ushiku, Yoshitaka Ono, Kanta |
| author_facet | Taniai, Tatsunori Igarashi, Ryo Suzuki, Yuta Chiba, Naoya Saito, Kotaro Ushiku, Yoshitaka Ono, Kanta |
| contents | Predicting physical properties of materials from their crystal structures is a fundamental problem in materials science. In peripheral areas such as the prediction of molecular properties, fully connected attention networks have been shown to be successful. However, unlike these finite atom arrangements, crystal structures are infinitely repeating, periodic arrangements of atoms, whose fully connected attention results in infinitely connected attention. In this work, we show that this infinitely connected attention can lead to a computationally tractable formulation, interpreted as neural potential summation, that performs infinite interatomic potential summations in a deeply learned feature space. We then propose a simple yet effective Transformer-based encoder architecture for crystal structures called Crystalformer. Compared to an existing Transformer-based model, the proposed model requires only 29.4% of the number of parameters, with minimal modifications to the original Transformer architecture. Despite the architectural simplicity, the proposed method outperforms state-of-the-art methods for various property regression tasks on the Materials Project and JARVIS-DFT datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_11686 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Crystalformer: Infinitely Connected Attention for Periodic Structure Encoding Taniai, Tatsunori Igarashi, Ryo Suzuki, Yuta Chiba, Naoya Saito, Kotaro Ushiku, Yoshitaka Ono, Kanta Machine Learning Materials Science Computational Physics Predicting physical properties of materials from their crystal structures is a fundamental problem in materials science. In peripheral areas such as the prediction of molecular properties, fully connected attention networks have been shown to be successful. However, unlike these finite atom arrangements, crystal structures are infinitely repeating, periodic arrangements of atoms, whose fully connected attention results in infinitely connected attention. In this work, we show that this infinitely connected attention can lead to a computationally tractable formulation, interpreted as neural potential summation, that performs infinite interatomic potential summations in a deeply learned feature space. We then propose a simple yet effective Transformer-based encoder architecture for crystal structures called Crystalformer. Compared to an existing Transformer-based model, the proposed model requires only 29.4% of the number of parameters, with minimal modifications to the original Transformer architecture. Despite the architectural simplicity, the proposed method outperforms state-of-the-art methods for various property regression tasks on the Materials Project and JARVIS-DFT datasets. |
| title | Crystalformer: Infinitely Connected Attention for Periodic Structure Encoding |
| topic | Machine Learning Materials Science Computational Physics |
| url | https://arxiv.org/abs/2403.11686 |