Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2410.06262 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929737197158400 |
|---|---|
| author | Zhang, Leo Ashouritaklimi, Kianoosh Teh, Yee Whye Cornish, Rob |
| author_facet | Zhang, Leo Ashouritaklimi, Kianoosh Teh, Yee Whye Cornish, Rob |
| contents | We propose SymDiff, a method for constructing equivariant diffusion models using the framework of stochastic symmetrisation. SymDiff resembles a learned data augmentation that is deployed at sampling time, and is lightweight, computationally efficient, and easy to implement on top of arbitrary off-the-shelf models. In contrast to previous work, SymDiff typically does not require any neural network components that are intrinsically equivariant, avoiding the need for complex parameterisations or the use of higher-order geometric features. Instead, our method can leverage highly scalable modern architectures as drop-in replacements for these more constrained alternatives. We show that this additional flexibility yields significant empirical benefit for $\mathrm{E}(3)$-equivariant molecular generation. To the best of our knowledge, this is the first application of symmetrisation to generative modelling, suggesting its potential in this domain more generally. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_06262 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | SymDiff: Equivariant Diffusion via Stochastic Symmetrisation Zhang, Leo Ashouritaklimi, Kianoosh Teh, Yee Whye Cornish, Rob Machine Learning We propose SymDiff, a method for constructing equivariant diffusion models using the framework of stochastic symmetrisation. SymDiff resembles a learned data augmentation that is deployed at sampling time, and is lightweight, computationally efficient, and easy to implement on top of arbitrary off-the-shelf models. In contrast to previous work, SymDiff typically does not require any neural network components that are intrinsically equivariant, avoiding the need for complex parameterisations or the use of higher-order geometric features. Instead, our method can leverage highly scalable modern architectures as drop-in replacements for these more constrained alternatives. We show that this additional flexibility yields significant empirical benefit for $\mathrm{E}(3)$-equivariant molecular generation. To the best of our knowledge, this is the first application of symmetrisation to generative modelling, suggesting its potential in this domain more generally. |
| title | SymDiff: Equivariant Diffusion via Stochastic Symmetrisation |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2410.06262 |