Token-Mol 1.0: Tokenized drug design with large language model
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910568766504960 |
|---|---|
| author | Wang, Jike Qin, Rui Wang, Mingyang Fang, Meijing Zhang, Yangyang Zhu, Yuchen Su, Qun Gou, Qiaolin Shen, Chao Zhang, Odin Wu, Zhenxing Jiang, Dejun Zhang, Xujun Zhao, Huifeng Wan, Xiaozhe Wu, Zhourui Liu, Liwei Kang, Yu Hsieh, Chang-Yu Hou, Tingjun |
| author_facet | Wang, Jike Qin, Rui Wang, Mingyang Fang, Meijing Zhang, Yangyang Zhu, Yuchen Su, Qun Gou, Qiaolin Shen, Chao Zhang, Odin Wu, Zhenxing Jiang, Dejun Zhang, Xujun Zhao, Huifeng Wan, Xiaozhe Wu, Zhourui Liu, Liwei Kang, Yu Hsieh, Chang-Yu Hou, Tingjun |
| contents | Significant interests have recently risen in leveraging sequence-based large language models (LLMs) for drug design. However, most current applications of LLMs in drug discovery lack the ability to comprehend three-dimensional (3D) structures, thereby limiting their effectiveness in tasks that explicitly involve molecular conformations. In this study, we introduced Token-Mol, a token-only 3D drug design model. This model encodes all molecular information, including 2D and 3D structures, as well as molecular property data, into tokens, which transforms classification and regression tasks in drug discovery into probabilistic prediction problems, thereby enabling learning through a unified paradigm. Token-Mol is built on the transformer decoder architecture and trained using random causal masking techniques. Additionally, we proposed the Gaussian cross-entropy (GCE) loss function to overcome the challenges in regression tasks, significantly enhancing the capacity of LLMs to learn continuous numerical values. Through a combination of fine-tuning and reinforcement learning (RL), Token-Mol achieves performance comparable to or surpassing existing task-specific methods across various downstream tasks, including pocket-based molecular generation, conformation generation, and molecular property prediction. Compared to existing molecular pre-trained models, Token-Mol exhibits superior proficiency in handling a wider range of downstream tasks essential for drug design. Notably, our approach improves regression task accuracy by approximately 30% compared to similar token-only methods. Token-Mol overcomes the precision limitations of token-only models and has the potential to integrate seamlessly with general models such as ChatGPT, paving the way for the development of a universal artificial intelligence drug design model that facilitates rapid and high-quality drug design by experts. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_07930 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Token-Mol 1.0: Tokenized drug design with large language model Wang, Jike Qin, Rui Wang, Mingyang Fang, Meijing Zhang, Yangyang Zhu, Yuchen Su, Qun Gou, Qiaolin Shen, Chao Zhang, Odin Wu, Zhenxing Jiang, Dejun Zhang, Xujun Zhao, Huifeng Wan, Xiaozhe Wu, Zhourui Liu, Liwei Kang, Yu Hsieh, Chang-Yu Hou, Tingjun Biomolecules Machine Learning Significant interests have recently risen in leveraging sequence-based large language models (LLMs) for drug design. However, most current applications of LLMs in drug discovery lack the ability to comprehend three-dimensional (3D) structures, thereby limiting their effectiveness in tasks that explicitly involve molecular conformations. In this study, we introduced Token-Mol, a token-only 3D drug design model. This model encodes all molecular information, including 2D and 3D structures, as well as molecular property data, into tokens, which transforms classification and regression tasks in drug discovery into probabilistic prediction problems, thereby enabling learning through a unified paradigm. Token-Mol is built on the transformer decoder architecture and trained using random causal masking techniques. Additionally, we proposed the Gaussian cross-entropy (GCE) loss function to overcome the challenges in regression tasks, significantly enhancing the capacity of LLMs to learn continuous numerical values. Through a combination of fine-tuning and reinforcement learning (RL), Token-Mol achieves performance comparable to or surpassing existing task-specific methods across various downstream tasks, including pocket-based molecular generation, conformation generation, and molecular property prediction. Compared to existing molecular pre-trained models, Token-Mol exhibits superior proficiency in handling a wider range of downstream tasks essential for drug design. Notably, our approach improves regression task accuracy by approximately 30% compared to similar token-only methods. Token-Mol overcomes the precision limitations of token-only models and has the potential to integrate seamlessly with general models such as ChatGPT, paving the way for the development of a universal artificial intelligence drug design model that facilitates rapid and high-quality drug design by experts. |
| title | Token-Mol 1.0: Tokenized drug design with large language model |
| topic | Biomolecules Machine Learning |
| url | https://arxiv.org/abs/2407.07930 |