Beyond Atoms: Enhancing Molecular Pretrained Representations with 3D Space Modeling
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916655924248576 |
|---|---|
| author | Lu, Shuqi Ji, Xiaohong Zhang, Bohang Yao, Lin Liu, Siyuan Gao, Zhifeng Zhang, Linfeng Ke, Guolin |
| author_facet | Lu, Shuqi Ji, Xiaohong Zhang, Bohang Yao, Lin Liu, Siyuan Gao, Zhifeng Zhang, Linfeng Ke, Guolin |
| contents | Molecular pretrained representations (MPR) has emerged as a powerful approach for addressing the challenge of limited supervised data in applications such as drug discovery and material design. While early MPR methods relied on 1D sequences and 2D graphs, recent advancements have incorporated 3D conformational information to capture rich atomic interactions. However, these prior models treat molecules merely as discrete atom sets, overlooking the space surrounding them. We argue from a physical perspective that only modeling these discrete points is insufficient. We first present a simple yet insightful observation: naively adding randomly sampled virtual points beyond atoms can surprisingly enhance MPR performance. In light of this, we propose a principled framework that incorporates the entire 3D space spanned by molecules. We implement the framework via a novel Transformer-based architecture, dubbed SpaceFormer, with three key components: (1) grid-based space discretization; (2) grid sampling/merging; and (3) efficient 3D positional encoding. Extensive experiments show that SpaceFormer significantly outperforms previous 3D MPR models across various downstream tasks with limited data, validating the benefit of leveraging the additional 3D space beyond atoms in MPR models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_10489 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Beyond Atoms: Enhancing Molecular Pretrained Representations with 3D Space Modeling Lu, Shuqi Ji, Xiaohong Zhang, Bohang Yao, Lin Liu, Siyuan Gao, Zhifeng Zhang, Linfeng Ke, Guolin Biomolecules Machine Learning Molecular pretrained representations (MPR) has emerged as a powerful approach for addressing the challenge of limited supervised data in applications such as drug discovery and material design. While early MPR methods relied on 1D sequences and 2D graphs, recent advancements have incorporated 3D conformational information to capture rich atomic interactions. However, these prior models treat molecules merely as discrete atom sets, overlooking the space surrounding them. We argue from a physical perspective that only modeling these discrete points is insufficient. We first present a simple yet insightful observation: naively adding randomly sampled virtual points beyond atoms can surprisingly enhance MPR performance. In light of this, we propose a principled framework that incorporates the entire 3D space spanned by molecules. We implement the framework via a novel Transformer-based architecture, dubbed SpaceFormer, with three key components: (1) grid-based space discretization; (2) grid sampling/merging; and (3) efficient 3D positional encoding. Extensive experiments show that SpaceFormer significantly outperforms previous 3D MPR models across various downstream tasks with limited data, validating the benefit of leveraging the additional 3D space beyond atoms in MPR models. |
| title | Beyond Atoms: Enhancing Molecular Pretrained Representations with 3D Space Modeling |
| topic | Biomolecules Machine Learning |
| url | https://arxiv.org/abs/2503.10489 |