Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2020
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2007.16012 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915780611801088 |
|---|---|
| author | Payne, Josh Srouji, Mario Yap, Dian Ang Kosaraju, Vineet |
| author_facet | Payne, Josh Srouji, Mario Yap, Dian Ang Kosaraju, Vineet |
| contents | Modern computational organic chemistry is becoming increasingly data-driven. There remain a large number of important unsolved problems in this area such as product prediction given reactants, drug discovery, and metric-optimized molecule synthesis, but efforts to solve these problems using machine learning have also increased in recent years. In this work, we propose the use of attention to study functional groups and other property-impacting molecular substructures from a data-driven perspective, using a transformer-based model (BERT) on datasets of string representations of molecules and analyzing the behavior of its attention heads. We then apply the representations of functional groups and atoms learned by the model to tackle problems of toxicity, solubility, drug-likeness, and synthesis accessibility on smaller datasets using the learned representations as features for graph convolution and attention models on the graph structure of molecules, as well as fine-tuning of BERT. Finally, we propose the use of attention visualization as a helpful tool for chemistry practitioners and students to quickly identify important substructures in various chemical properties. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2007_16012 |
| institution | arXiv |
| publishDate | 2020 |
| record_format | arxiv |
| spellingShingle | BERT Learns (and Teaches) Chemistry Payne, Josh Srouji, Mario Yap, Dian Ang Kosaraju, Vineet Biomolecules Artificial Intelligence Computation and Language Machine Learning Modern computational organic chemistry is becoming increasingly data-driven. There remain a large number of important unsolved problems in this area such as product prediction given reactants, drug discovery, and metric-optimized molecule synthesis, but efforts to solve these problems using machine learning have also increased in recent years. In this work, we propose the use of attention to study functional groups and other property-impacting molecular substructures from a data-driven perspective, using a transformer-based model (BERT) on datasets of string representations of molecules and analyzing the behavior of its attention heads. We then apply the representations of functional groups and atoms learned by the model to tackle problems of toxicity, solubility, drug-likeness, and synthesis accessibility on smaller datasets using the learned representations as features for graph convolution and attention models on the graph structure of molecules, as well as fine-tuning of BERT. Finally, we propose the use of attention visualization as a helpful tool for chemistry practitioners and students to quickly identify important substructures in various chemical properties. |
| title | BERT Learns (and Teaches) Chemistry |
| topic | Biomolecules Artificial Intelligence Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2007.16012 |