Best Practices for Biorisk Evaluations on Open-Weight Bio-Foundation Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908668009644032 |
|---|---|
| author | Wei, Boyi Che, Zora Li, Nathaniel Sehwag, Udari Madhushani Götting, Jasper Nedungadi, Samira Michael, Julian Yue, Summer Hendrycks, Dan Henderson, Peter Wang, Zifan Donoughe, Seth Mazeika, Mantas |
| author_facet | Wei, Boyi Che, Zora Li, Nathaniel Sehwag, Udari Madhushani Götting, Jasper Nedungadi, Samira Michael, Julian Yue, Summer Hendrycks, Dan Henderson, Peter Wang, Zifan Donoughe, Seth Mazeika, Mantas |
| contents | Open-weight bio-foundation models present a dual-use dilemma. While holding great promise for accelerating scientific research and drug development, they could also enable bad actors to develop more deadly bioweapons. To mitigate the risk posed by these models, current approaches focus on filtering biohazardous data during pre-training. However, the effectiveness of such an approach remains unclear, particularly against determined actors who might fine-tune these models for malicious use. To address this gap, we propose BioRiskEval, a framework to evaluate the robustness of procedures that are intended to reduce the dual-use capabilities of bio-foundation models. BioRiskEval assesses models' virus understanding through three lenses, including sequence modeling, mutational effects prediction, and virulence prediction. Our results show that current filtering practices may not be particularly effective: Excluded knowledge can be rapidly recovered in some cases via fine-tuning, and exhibits broader generalizability in sequence modeling. Furthermore, dual-use signals may already reside in the pretrained representations, and can be elicited via simple linear probing. These findings highlight the challenges of data filtering as a standalone procedure, underscoring the need for further research into robust safety and security strategies for open-weight bio-foundation models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_27629 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Best Practices for Biorisk Evaluations on Open-Weight Bio-Foundation Models Wei, Boyi Che, Zora Li, Nathaniel Sehwag, Udari Madhushani Götting, Jasper Nedungadi, Samira Michael, Julian Yue, Summer Hendrycks, Dan Henderson, Peter Wang, Zifan Donoughe, Seth Mazeika, Mantas Cryptography and Security Artificial Intelligence Open-weight bio-foundation models present a dual-use dilemma. While holding great promise for accelerating scientific research and drug development, they could also enable bad actors to develop more deadly bioweapons. To mitigate the risk posed by these models, current approaches focus on filtering biohazardous data during pre-training. However, the effectiveness of such an approach remains unclear, particularly against determined actors who might fine-tune these models for malicious use. To address this gap, we propose BioRiskEval, a framework to evaluate the robustness of procedures that are intended to reduce the dual-use capabilities of bio-foundation models. BioRiskEval assesses models' virus understanding through three lenses, including sequence modeling, mutational effects prediction, and virulence prediction. Our results show that current filtering practices may not be particularly effective: Excluded knowledge can be rapidly recovered in some cases via fine-tuning, and exhibits broader generalizability in sequence modeling. Furthermore, dual-use signals may already reside in the pretrained representations, and can be elicited via simple linear probing. These findings highlight the challenges of data filtering as a standalone procedure, underscoring the need for further research into robust safety and security strategies for open-weight bio-foundation models. |
| title | Best Practices for Biorisk Evaluations on Open-Weight Bio-Foundation Models |
| topic | Cryptography and Security Artificial Intelligence |
| url | https://arxiv.org/abs/2510.27629 |