Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2603.06740 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911698095439872 |
|---|---|
| author | Zhou, Yichen Golob, Jonathan Karimi, Amir Bauer, Stefan Schwab, Patrick |
| author_facet | Zhou, Yichen Golob, Jonathan Karimi, Amir Bauer, Stefan Schwab, Patrick |
| contents | Protein language models (pLMs) have shown strong potential for zero-shot prediction of missense variant effects, yet systematic benchmarking on viral proteins remains limited, a critical gap given the need for proactive tools that can anticipate emerging mutations ahead of experimental validation. Here we introduce ViroGym, a comprehensive benchmark evaluating pLMs across three tasks: 79 deep mutational scanning (DMS) assays covering eukaryotic viruses with 552,065 mutated sequences across 7 phenotypic readouts, 21 influenza neutralisation tasks, and a real-world pandemic prediction task for SARS-CoV-2. We benchmark well-established pLMs on fitness landscapes, antigenic diversity, and pandemic forecasting, and find that the ProGen2 family consistently achieves the strongest performance across all three tasks. Crucially, DMS and neutralisation performance reliably identifies models that generalise to real-world emergence, even though the mutation sets they surface barely overlap, revealing that complementary in vitro benchmarks capture the evolutionary constraints needed for real-world mutation forecasting. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_06740 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | ViroGym: Realistic Large-Scale Benchmarks for Evaluating Viral Proteins Zhou, Yichen Golob, Jonathan Karimi, Amir Bauer, Stefan Schwab, Patrick Quantitative Methods Artificial Intelligence Protein language models (pLMs) have shown strong potential for zero-shot prediction of missense variant effects, yet systematic benchmarking on viral proteins remains limited, a critical gap given the need for proactive tools that can anticipate emerging mutations ahead of experimental validation. Here we introduce ViroGym, a comprehensive benchmark evaluating pLMs across three tasks: 79 deep mutational scanning (DMS) assays covering eukaryotic viruses with 552,065 mutated sequences across 7 phenotypic readouts, 21 influenza neutralisation tasks, and a real-world pandemic prediction task for SARS-CoV-2. We benchmark well-established pLMs on fitness landscapes, antigenic diversity, and pandemic forecasting, and find that the ProGen2 family consistently achieves the strongest performance across all three tasks. Crucially, DMS and neutralisation performance reliably identifies models that generalise to real-world emergence, even though the mutation sets they surface barely overlap, revealing that complementary in vitro benchmarks capture the evolutionary constraints needed for real-world mutation forecasting. |
| title | ViroGym: Realistic Large-Scale Benchmarks for Evaluating Viral Proteins |
| topic | Quantitative Methods Artificial Intelligence |
| url | https://arxiv.org/abs/2603.06740 |