nach0-pc: Multi-task Language Model with Molecular Point Cloud Encoder
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916434951536640 |
|---|---|
| author | Kuznetsov, Maksim Valiev, Airat Aliper, Alex Polykovskiy, Daniil Tutubalina, Elena Shayakhmetov, Rim Miftahutdinov, Zulfat |
| author_facet | Kuznetsov, Maksim Valiev, Airat Aliper, Alex Polykovskiy, Daniil Tutubalina, Elena Shayakhmetov, Rim Miftahutdinov, Zulfat |
| contents | Recent advancements have integrated Language Models (LMs) into a drug discovery pipeline. However, existing models mostly work with SMILES and SELFIES chemical string representations, which lack spatial features vital for drug discovery. Additionally, attempts to translate chemical 3D structures into text format encounter issues such as excessive length and insufficient atom connectivity information. To address these issues, we introduce nach0-pc, a model combining domain-specific encoder and textual representation to handle spatial arrangement of atoms effectively. Our approach utilizes a molecular point cloud encoder for concise and order-invariant structure representation. We introduce a novel pre-training scheme for molecular point clouds to distillate the knowledge from spatial molecular structures datasets. After fine-tuning within both single-task and multi-task frameworks, nach0-pc demonstrates performance comparable with other diffusion models in terms of generated samples quality across several established spatial molecular generation tasks. Notably, our model is a multi-task approach, in contrast to diffusion models being limited to single tasks. Additionally, it is capable of processing point cloud-related data, which language models are not capable of handling due to memory limitations. These lead to our model having reduced training and inference time while maintaining on par performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_09240 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | nach0-pc: Multi-task Language Model with Molecular Point Cloud Encoder Kuznetsov, Maksim Valiev, Airat Aliper, Alex Polykovskiy, Daniil Tutubalina, Elena Shayakhmetov, Rim Miftahutdinov, Zulfat Machine Learning Computation and Language Recent advancements have integrated Language Models (LMs) into a drug discovery pipeline. However, existing models mostly work with SMILES and SELFIES chemical string representations, which lack spatial features vital for drug discovery. Additionally, attempts to translate chemical 3D structures into text format encounter issues such as excessive length and insufficient atom connectivity information. To address these issues, we introduce nach0-pc, a model combining domain-specific encoder and textual representation to handle spatial arrangement of atoms effectively. Our approach utilizes a molecular point cloud encoder for concise and order-invariant structure representation. We introduce a novel pre-training scheme for molecular point clouds to distillate the knowledge from spatial molecular structures datasets. After fine-tuning within both single-task and multi-task frameworks, nach0-pc demonstrates performance comparable with other diffusion models in terms of generated samples quality across several established spatial molecular generation tasks. Notably, our model is a multi-task approach, in contrast to diffusion models being limited to single tasks. Additionally, it is capable of processing point cloud-related data, which language models are not capable of handling due to memory limitations. These lead to our model having reduced training and inference time while maintaining on par performance. |
| title | nach0-pc: Multi-task Language Model with Molecular Point Cloud Encoder |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2410.09240 |