Distillation of atomistic foundation models across architectures and chemical domains
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916792186699776 |
|---|---|
| author | Gardner, John L. A. Toit, Daniel F. Thomas du Mahmoud, Chiheb Ben Beaulieu, Zoé Faure Juraskova, Veronika Paşca, Laura-Bianca Rosset, Louise A. M. Duarte, Fernanda Martelli, Fausto Pickard, Chris J. Deringer, Volker L. |
| author_facet | Gardner, John L. A. Toit, Daniel F. Thomas du Mahmoud, Chiheb Ben Beaulieu, Zoé Faure Juraskova, Veronika Paşca, Laura-Bianca Rosset, Louise A. M. Duarte, Fernanda Martelli, Fausto Pickard, Chris J. Deringer, Volker L. |
| contents | Machine-learned interatomic potentials have transformed computational research in the physical sciences. Recent atomistic `foundation' models have changed the field yet again: trained on many different chemical elements and domains, these potentials are widely applicable, but comparably slow and resource-intensive to run. Here we show how distillation via synthetic data can be used to cheaply transfer knowledge from atomistic foundation models to a range of different architectures, unlocking much smaller, more efficient potentials. We demonstrate speed-ups of $> 10\times$ by distilling from one graph-network architecture into another, and $> 100\times$ by leveraging the atomic cluster expansion framework. We showcase applicability across chemical and materials domains: from liquid water to hydrogen under extreme conditions; from porous silica and a hybrid halide perovskite solar-cell material to modelling organic reactions. Our work shows how distillation can support the routine and computationally efficient use of current and future atomistic foundation models in real-world scientific research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_10956 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Distillation of atomistic foundation models across architectures and chemical domains Gardner, John L. A. Toit, Daniel F. Thomas du Mahmoud, Chiheb Ben Beaulieu, Zoé Faure Juraskova, Veronika Paşca, Laura-Bianca Rosset, Louise A. M. Duarte, Fernanda Martelli, Fausto Pickard, Chris J. Deringer, Volker L. Computational Physics Machine-learned interatomic potentials have transformed computational research in the physical sciences. Recent atomistic `foundation' models have changed the field yet again: trained on many different chemical elements and domains, these potentials are widely applicable, but comparably slow and resource-intensive to run. Here we show how distillation via synthetic data can be used to cheaply transfer knowledge from atomistic foundation models to a range of different architectures, unlocking much smaller, more efficient potentials. We demonstrate speed-ups of $> 10\times$ by distilling from one graph-network architecture into another, and $> 100\times$ by leveraging the atomic cluster expansion framework. We showcase applicability across chemical and materials domains: from liquid water to hydrogen under extreme conditions; from porous silica and a hybrid halide perovskite solar-cell material to modelling organic reactions. Our work shows how distillation can support the routine and computationally efficient use of current and future atomistic foundation models in real-world scientific research. |
| title | Distillation of atomistic foundation models across architectures and chemical domains |
| topic | Computational Physics |
| url | https://arxiv.org/abs/2506.10956 |