Difficulty in chirality recognition for Transformer architectures learning chemical structures from string
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910335512870912 |
|---|---|
| author | Yoshikai, Yasuhiro Mizuno, Tadahaya Nemoto, Shumpei Kusuhara, Hiroyuki |
| author_facet | Yoshikai, Yasuhiro Mizuno, Tadahaya Nemoto, Shumpei Kusuhara, Hiroyuki |
| contents | Recent years have seen rapid development of descriptor generation based on representation learning of extremely diverse molecules, especially those that apply natural language processing (NLP) models to SMILES, a literal representation of molecular structure. However, little research has been done on how these models understand chemical structure. To address this black box, we investigated the relationship between the learning progress of SMILES and chemical structure using a representative NLP model, the Transformer. We show that while the Transformer learns partial structures of molecules quickly, it requires extended training to understand overall structures. Consistently, the accuracy of molecular property predictions using descriptors generated from models at different learning steps was similar from the beginning to the end of training. Furthermore, we found that the Transformer requires particularly long training to learn chirality and sometimes stagnates with low performance due to misunderstanding of enantiomers. These findings are expected to deepen the understanding of NLP models in chemistry. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2303_11593 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Difficulty in chirality recognition for Transformer architectures learning chemical structures from string Yoshikai, Yasuhiro Mizuno, Tadahaya Nemoto, Shumpei Kusuhara, Hiroyuki Machine Learning Computation and Language Chemical Physics Biomolecules J.2; I.2.7 Recent years have seen rapid development of descriptor generation based on representation learning of extremely diverse molecules, especially those that apply natural language processing (NLP) models to SMILES, a literal representation of molecular structure. However, little research has been done on how these models understand chemical structure. To address this black box, we investigated the relationship between the learning progress of SMILES and chemical structure using a representative NLP model, the Transformer. We show that while the Transformer learns partial structures of molecules quickly, it requires extended training to understand overall structures. Consistently, the accuracy of molecular property predictions using descriptors generated from models at different learning steps was similar from the beginning to the end of training. Furthermore, we found that the Transformer requires particularly long training to learn chirality and sometimes stagnates with low performance due to misunderstanding of enantiomers. These findings are expected to deepen the understanding of NLP models in chemistry. |
| title | Difficulty in chirality recognition for Transformer architectures learning chemical structures from string |
| topic | Machine Learning Computation and Language Chemical Physics Biomolecules J.2; I.2.7 |
| url | https://arxiv.org/abs/2303.11593 |