Comprehensive Molecular Representation from Equivariant Transformer
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2023
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911790632271872 |
|---|---|
| author | Tao, Nianze Morimoto, Hiromi Leoni, Stefano |
| author_facet | Tao, Nianze Morimoto, Hiromi Leoni, Stefano |
| contents | The tradeoff between precision and performance in molecular simulations can nowadays be addressed by machine-learned force fields (MLFF), which combine \textit{ab initio} accuracy with force field numerical efficiency. Different from conventional force fields however, incorporating relevant electronic degrees of freedom into MLFFs becomes important. Here, we implement an equivariant transformer that embeds molecular net charge and spin state without additional neural network parameters. The model trained on a singlet/triplet non-correlated \ce{CH2} dataset can identify different spin states and shows state-of-the-art extrapolation capability. Therein, self-attention sensibly captures non-local effects, which, as we show, can be finely tuned over the network hyper-parameters. We indeed found that Softmax activation functions utilised in the self-attention mechanism of graph networks outperformed ReLU-like functions in prediction accuracy. Increasing the attention temperature from $τ= \sqrt{d}$ to $\sqrt{2d}$ further improved the extrapolation capability, indicating a weighty role of nonlocality. Additionally, a weight initialisation method was purposed that sensibly accelerated the training process. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2308_10752 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Comprehensive Molecular Representation from Equivariant Transformer Tao, Nianze Morimoto, Hiromi Leoni, Stefano Computational Physics Materials Science Atomic and Molecular Clusters Chemical Physics The tradeoff between precision and performance in molecular simulations can nowadays be addressed by machine-learned force fields (MLFF), which combine \textit{ab initio} accuracy with force field numerical efficiency. Different from conventional force fields however, incorporating relevant electronic degrees of freedom into MLFFs becomes important. Here, we implement an equivariant transformer that embeds molecular net charge and spin state without additional neural network parameters. The model trained on a singlet/triplet non-correlated \ce{CH2} dataset can identify different spin states and shows state-of-the-art extrapolation capability. Therein, self-attention sensibly captures non-local effects, which, as we show, can be finely tuned over the network hyper-parameters. We indeed found that Softmax activation functions utilised in the self-attention mechanism of graph networks outperformed ReLU-like functions in prediction accuracy. Increasing the attention temperature from $τ= \sqrt{d}$ to $\sqrt{2d}$ further improved the extrapolation capability, indicating a weighty role of nonlocality. Additionally, a weight initialisation method was purposed that sensibly accelerated the training process. |
| title | Comprehensive Molecular Representation from Equivariant Transformer |
| topic | Computational Physics Materials Science Atomic and Molecular Clusters Chemical Physics |
| url | https://arxiv.org/abs/2308.10752 |