ESM All-Atom: Multi-scale Protein Language Model for Unified Molecular Modeling
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911915163254784 |
|---|---|
| author | Zheng, Kangjie Long, Siyu Lu, Tianyu Yang, Junwei Dai, Xinyu Zhang, Ming Nie, Zaiqing Ma, Wei-Ying Zhou, Hao |
| author_facet | Zheng, Kangjie Long, Siyu Lu, Tianyu Yang, Junwei Dai, Xinyu Zhang, Ming Nie, Zaiqing Ma, Wei-Ying Zhou, Hao |
| contents | Protein language models have demonstrated significant potential in the field of protein engineering. However, current protein language models primarily operate at the residue scale, which limits their ability to provide information at the atom level. This limitation prevents us from fully exploiting the capabilities of protein language models for applications involving both proteins and small molecules. In this paper, we propose ESM-AA (ESM All-Atom), a novel approach that enables atom-scale and residue-scale unified molecular modeling. ESM-AA achieves this by pre-training on multi-scale code-switch protein sequences and utilizing a multi-scale position encoding to capture relationships among residues and atoms. Experimental results indicate that ESM-AA surpasses previous methods in protein-molecule tasks, demonstrating the full utilization of protein language models. Further investigations reveal that through unified molecular modeling, ESM-AA not only gains molecular knowledge but also retains its understanding of proteins. The source codes of ESM-AA are publicly released at https://github.com/zhengkangjie/ESM-AA. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_12995 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ESM All-Atom: Multi-scale Protein Language Model for Unified Molecular Modeling Zheng, Kangjie Long, Siyu Lu, Tianyu Yang, Junwei Dai, Xinyu Zhang, Ming Nie, Zaiqing Ma, Wei-Ying Zhou, Hao Biomolecules Computational Engineering, Finance, and Science Machine Learning Protein language models have demonstrated significant potential in the field of protein engineering. However, current protein language models primarily operate at the residue scale, which limits their ability to provide information at the atom level. This limitation prevents us from fully exploiting the capabilities of protein language models for applications involving both proteins and small molecules. In this paper, we propose ESM-AA (ESM All-Atom), a novel approach that enables atom-scale and residue-scale unified molecular modeling. ESM-AA achieves this by pre-training on multi-scale code-switch protein sequences and utilizing a multi-scale position encoding to capture relationships among residues and atoms. Experimental results indicate that ESM-AA surpasses previous methods in protein-molecule tasks, demonstrating the full utilization of protein language models. Further investigations reveal that through unified molecular modeling, ESM-AA not only gains molecular knowledge but also retains its understanding of proteins. The source codes of ESM-AA are publicly released at https://github.com/zhengkangjie/ESM-AA. |
| title | ESM All-Atom: Multi-scale Protein Language Model for Unified Molecular Modeling |
| topic | Biomolecules Computational Engineering, Finance, and Science Machine Learning |
| url | https://arxiv.org/abs/2403.12995 |