A recipe for scalable attention-based MLIPs: unlocking long-range accuracy with all-to-all node attention
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914376064172032 |
|---|---|
| author | Qu, Eric Wood, Brandon M. Krishnapriyan, Aditi S. Ulissi, Zachary W. |
| author_facet | Qu, Eric Wood, Brandon M. Krishnapriyan, Aditi S. Ulissi, Zachary W. |
| contents | Machine-learning interatomic potentials (MLIPs) have advanced rapidly, with many top models relying on strong physics-based inductive biases. However, as models scale to larger systems like biomolecules and electrolytes, they struggle to accurately capture long-range (LR) interactions, leading current approaches to rely on explicit physics-based terms or components. In this work, we propose AllScAIP, a straightforward, attention-based, and energy-conserving MLIP model that scales to O(100 million) training samples. It addresses the long-range challenge using an all-to-all node attention component that is data-driven. Extensive ablations reveal that in low-data/small-model regimes, inductive biases improve sample efficiency. However, as data and model size scale, these benefits diminish or even reverse, while all-to-all attention remains critical for capturing LR interactions. Our model achieves state-of-the-art energy/force accuracy on molecular systems, as well as a number of physics-based evaluations (OMol25), while being competitive on materials (OMat24) and catalysts (OC20). Furthermore, it enables stable, long-timescale MD simulations that accurately recover experimental observables, including density and heat of vaporization predictions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_06567 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | A recipe for scalable attention-based MLIPs: unlocking long-range accuracy with all-to-all node attention Qu, Eric Wood, Brandon M. Krishnapriyan, Aditi S. Ulissi, Zachary W. Machine Learning Materials Science Computational Engineering, Finance, and Science Chemical Physics Quantitative Methods Machine-learning interatomic potentials (MLIPs) have advanced rapidly, with many top models relying on strong physics-based inductive biases. However, as models scale to larger systems like biomolecules and electrolytes, they struggle to accurately capture long-range (LR) interactions, leading current approaches to rely on explicit physics-based terms or components. In this work, we propose AllScAIP, a straightforward, attention-based, and energy-conserving MLIP model that scales to O(100 million) training samples. It addresses the long-range challenge using an all-to-all node attention component that is data-driven. Extensive ablations reveal that in low-data/small-model regimes, inductive biases improve sample efficiency. However, as data and model size scale, these benefits diminish or even reverse, while all-to-all attention remains critical for capturing LR interactions. Our model achieves state-of-the-art energy/force accuracy on molecular systems, as well as a number of physics-based evaluations (OMol25), while being competitive on materials (OMat24) and catalysts (OC20). Furthermore, it enables stable, long-timescale MD simulations that accurately recover experimental observables, including density and heat of vaporization predictions. |
| title | A recipe for scalable attention-based MLIPs: unlocking long-range accuracy with all-to-all node attention |
| topic | Machine Learning Materials Science Computational Engineering, Finance, and Science Chemical Physics Quantitative Methods |
| url | https://arxiv.org/abs/2603.06567 |