BOOM: Benchmarking Out-Of-distribution Molecular Property Predictions of Machine Learning Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911328647512064 |
|---|---|
| author | Antoniuk, Evan R. Zaman, Shehtab Ben-Nun, Tal Li, Peggy Diffenderfer, James Sahin, Busra Smolenski, Obadiah Hsu, Tim Hiszpanski, Anna M. Chiu, Kenneth Kailkhura, Bhavya Van Essen, Brian |
| author_facet | Antoniuk, Evan R. Zaman, Shehtab Ben-Nun, Tal Li, Peggy Diffenderfer, James Sahin, Busra Smolenski, Obadiah Hsu, Tim Hiszpanski, Anna M. Chiu, Kenneth Kailkhura, Bhavya Van Essen, Brian |
| contents | Data-driven molecular discovery leverages artificial intelligence/machine learning (AI/ML) and generative modeling to filter and design novel molecules. Discovering novel molecules requires accurate out-of-distribution (OOD) predictions, but ML models struggle to generalize OOD. Currently, no systematic benchmarks exist for molecular OOD prediction tasks. We present $\mathbf{BOOM}$, $\mathbf{b}$enchmarks for $\mathbf{o}$ut-$\mathbf{o}$f-distribution $\mathbf{m}$olecular property predictions: a chemically-informed benchmark for OOD performance on common molecular property prediction tasks. We evaluate over 150 model-task combinations to benchmark deep learning models on OOD performance. Overall, we find that no existing model achieves strong generalization across all tasks: even the top-performing model exhibited an average OOD error 3x higher than in-distribution. Current chemical foundation models do not show strong OOD extrapolation, while models with high inductive bias can perform well on OOD tasks with simple, specific properties. We perform extensive ablation experiments, highlighting how data generation, pre-training, hyperparameter optimization, model architecture, and molecular representation impact OOD performance. Developing models with strong OOD generalization is a new frontier challenge in chemical ML. This open-source benchmark is available at https://github.com/FLASK-LLNL/BOOM |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_01912 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | BOOM: Benchmarking Out-Of-distribution Molecular Property Predictions of Machine Learning Models Antoniuk, Evan R. Zaman, Shehtab Ben-Nun, Tal Li, Peggy Diffenderfer, James Sahin, Busra Smolenski, Obadiah Hsu, Tim Hiszpanski, Anna M. Chiu, Kenneth Kailkhura, Bhavya Van Essen, Brian Machine Learning Materials Science Artificial Intelligence Data-driven molecular discovery leverages artificial intelligence/machine learning (AI/ML) and generative modeling to filter and design novel molecules. Discovering novel molecules requires accurate out-of-distribution (OOD) predictions, but ML models struggle to generalize OOD. Currently, no systematic benchmarks exist for molecular OOD prediction tasks. We present $\mathbf{BOOM}$, $\mathbf{b}$enchmarks for $\mathbf{o}$ut-$\mathbf{o}$f-distribution $\mathbf{m}$olecular property predictions: a chemically-informed benchmark for OOD performance on common molecular property prediction tasks. We evaluate over 150 model-task combinations to benchmark deep learning models on OOD performance. Overall, we find that no existing model achieves strong generalization across all tasks: even the top-performing model exhibited an average OOD error 3x higher than in-distribution. Current chemical foundation models do not show strong OOD extrapolation, while models with high inductive bias can perform well on OOD tasks with simple, specific properties. We perform extensive ablation experiments, highlighting how data generation, pre-training, hyperparameter optimization, model architecture, and molecular representation impact OOD performance. Developing models with strong OOD generalization is a new frontier challenge in chemical ML. This open-source benchmark is available at https://github.com/FLASK-LLNL/BOOM |
| title | BOOM: Benchmarking Out-Of-distribution Molecular Property Predictions of Machine Learning Models |
| topic | Machine Learning Materials Science Artificial Intelligence |
| url | https://arxiv.org/abs/2505.01912 |