PFMBench: Protein Foundation Model Benchmark
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915348710686720 |
|---|---|
| author | Gao, Zhangyang Wang, Hao Tan, Cheng Xu, Chenrui Liu, Mengdi Hu, Bozhen Chao, Linlin Zhang, Xiaoming Li, Stan Z. |
| author_facet | Gao, Zhangyang Wang, Hao Tan, Cheng Xu, Chenrui Liu, Mengdi Hu, Bozhen Chao, Linlin Zhang, Xiaoming Li, Stan Z. |
| contents | This study investigates the current landscape and future directions of protein foundation model research. While recent advancements have transformed protein science and engineering, the field lacks a comprehensive benchmark for fair evaluation and in-depth understanding. Since ESM-1B, numerous protein foundation models have emerged, each with unique datasets and methodologies. However, evaluations often focus on limited tasks tailored to specific models, hindering insights into broader generalization and limitations. Specifically, researchers struggle to understand the relationships between tasks, assess how well current models perform across them, and determine the criteria in developing new foundation models. To fill this gap, we present PFMBench, a comprehensive benchmark evaluating protein foundation models across 38 tasks spanning 8 key areas of protein science. Through hundreds of experiments on 17 state-of-the-art models across 38 tasks, PFMBench reveals the inherent correlations between tasks, identifies top-performing models, and provides a streamlined evaluation protocol. Code is available at \href{https://github.com/biomap-research/PFMBench}{\textcolor{blue}{GitHub}}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_14796 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | PFMBench: Protein Foundation Model Benchmark Gao, Zhangyang Wang, Hao Tan, Cheng Xu, Chenrui Liu, Mengdi Hu, Bozhen Chao, Linlin Zhang, Xiaoming Li, Stan Z. Biomolecules Artificial Intelligence Machine Learning This study investigates the current landscape and future directions of protein foundation model research. While recent advancements have transformed protein science and engineering, the field lacks a comprehensive benchmark for fair evaluation and in-depth understanding. Since ESM-1B, numerous protein foundation models have emerged, each with unique datasets and methodologies. However, evaluations often focus on limited tasks tailored to specific models, hindering insights into broader generalization and limitations. Specifically, researchers struggle to understand the relationships between tasks, assess how well current models perform across them, and determine the criteria in developing new foundation models. To fill this gap, we present PFMBench, a comprehensive benchmark evaluating protein foundation models across 38 tasks spanning 8 key areas of protein science. Through hundreds of experiments on 17 state-of-the-art models across 38 tasks, PFMBench reveals the inherent correlations between tasks, identifies top-performing models, and provides a streamlined evaluation protocol. Code is available at \href{https://github.com/biomap-research/PFMBench}{\textcolor{blue}{GitHub}}. |
| title | PFMBench: Protein Foundation Model Benchmark |
| topic | Biomolecules Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2506.14796 |