One protein is all you need
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915824848076800 |
|---|---|
| author | Bushuiev, Anton Bushuiev, Roman Pimenova, Olga Zadorozhny, Nikola Samusevich, Raman Manaskova, Elisabet Kim, Rachel Seongeun Stärk, Hannes Sedlar, Jiri Steinegger, Martin Pluskal, Tomáš Sivic, Josef |
| author_facet | Bushuiev, Anton Bushuiev, Roman Pimenova, Olga Zadorozhny, Nikola Samusevich, Raman Manaskova, Elisabet Kim, Rachel Seongeun Stärk, Hannes Sedlar, Jiri Steinegger, Martin Pluskal, Tomáš Sivic, Josef |
| contents | Generalization beyond training data remains a central challenge in machine learning for biology. A common way to enhance generalization is self-supervised pre-training on large datasets. However, aiming to perform well on all possible proteins can limit a model's capacity to excel on any specific one, whereas experimentalists typically need accurate predictions for individual proteins they study, often not covered in training data. To address this limitation, we propose a method that enables self-supervised customization of protein language models to one target protein at a time, on the fly, and without assuming any additional data. We show that our Protein Test-Time Training (ProteinTTT) method consistently enhances generalization across different models, their sizes, and datasets. ProteinTTT improves structure prediction for challenging targets, achieves new state-of-the-art results on protein fitness prediction, and enhances function prediction on two tasks. Through two challenging case studies, we also show that customization via ProteinTTT achieves more accurate antibody-antigen loop modeling and enhances 19% of structures in the Big Fantastic Virus Database, delivering improved predictions where general-purpose AlphaFold2 and ESMFold struggle. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_02109 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | One protein is all you need Bushuiev, Anton Bushuiev, Roman Pimenova, Olga Zadorozhny, Nikola Samusevich, Raman Manaskova, Elisabet Kim, Rachel Seongeun Stärk, Hannes Sedlar, Jiri Steinegger, Martin Pluskal, Tomáš Sivic, Josef Machine Learning Biomolecules Generalization beyond training data remains a central challenge in machine learning for biology. A common way to enhance generalization is self-supervised pre-training on large datasets. However, aiming to perform well on all possible proteins can limit a model's capacity to excel on any specific one, whereas experimentalists typically need accurate predictions for individual proteins they study, often not covered in training data. To address this limitation, we propose a method that enables self-supervised customization of protein language models to one target protein at a time, on the fly, and without assuming any additional data. We show that our Protein Test-Time Training (ProteinTTT) method consistently enhances generalization across different models, their sizes, and datasets. ProteinTTT improves structure prediction for challenging targets, achieves new state-of-the-art results on protein fitness prediction, and enhances function prediction on two tasks. Through two challenging case studies, we also show that customization via ProteinTTT achieves more accurate antibody-antigen loop modeling and enhances 19% of structures in the Big Fantastic Virus Database, delivering improved predictions where general-purpose AlphaFold2 and ESMFold struggle. |
| title | One protein is all you need |
| topic | Machine Learning Biomolecules |
| url | https://arxiv.org/abs/2411.02109 |