Beyond ESM2: Graph-Enhanced Protein Sequence Modeling with Efficient Clustering
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917648789405696 |
|---|---|
| author | Jiao, Shujian Li, Bingxuan Wang, Lei Zhang, Xiaojin Chen, Wei Peng, Jiajie Wei, Zhongyu |
| author_facet | Jiao, Shujian Li, Bingxuan Wang, Lei Zhang, Xiaojin Chen, Wei Peng, Jiajie Wei, Zhongyu |
| contents | Proteins are essential to life's processes, underpinning evolution and diversity. Advances in sequencing technology have revealed millions of proteins, underscoring the need for sophisticated pre-trained protein models for biological analysis and AI development. Facebook's ESM2, the most advanced protein language model to date, leverages a masked prediction task for unsupervised learning, crafting amino acid representations with notable biochemical accuracy. Yet, it lacks in delivering functional protein insights, signaling an opportunity for enhancing representation quality.Our study addresses this gap by incorporating protein family classification into ESM2's training.This approach, augmented with Community Propagation-Based Clustering Algorithm, improves global protein representations, while a contextual prediction task fine-tunes local amino acid accuracy. Significantly, our model achieved state-of-the-art results in several downstream experiments, demonstrating the power of combining global and local methodologies to substantially boost protein representation quality. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_15805 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Beyond ESM2: Graph-Enhanced Protein Sequence Modeling with Efficient Clustering Jiao, Shujian Li, Bingxuan Wang, Lei Zhang, Xiaojin Chen, Wei Peng, Jiajie Wei, Zhongyu Biomolecules Machine Learning Proteins are essential to life's processes, underpinning evolution and diversity. Advances in sequencing technology have revealed millions of proteins, underscoring the need for sophisticated pre-trained protein models for biological analysis and AI development. Facebook's ESM2, the most advanced protein language model to date, leverages a masked prediction task for unsupervised learning, crafting amino acid representations with notable biochemical accuracy. Yet, it lacks in delivering functional protein insights, signaling an opportunity for enhancing representation quality.Our study addresses this gap by incorporating protein family classification into ESM2's training.This approach, augmented with Community Propagation-Based Clustering Algorithm, improves global protein representations, while a contextual prediction task fine-tunes local amino acid accuracy. Significantly, our model achieved state-of-the-art results in several downstream experiments, demonstrating the power of combining global and local methodologies to substantially boost protein representation quality. |
| title | Beyond ESM2: Graph-Enhanced Protein Sequence Modeling with Efficient Clustering |
| topic | Biomolecules Machine Learning |
| url | https://arxiv.org/abs/2404.15805 |