Self-Supervised Learning with Lie Symmetries for Partial Differential Equations
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917589505015808 |
|---|---|
| author | Mialon, Grégoire Garrido, Quentin Lawrence, Hannah Rehman, Danyal LeCun, Yann Kiani, Bobak T. |
| author_facet | Mialon, Grégoire Garrido, Quentin Lawrence, Hannah Rehman, Danyal LeCun, Yann Kiani, Bobak T. |
| contents | Machine learning for differential equations paves the way for computationally efficient alternatives to numerical solvers, with potentially broad impacts in science and engineering. Though current algorithms typically require simulated training data tailored to a given setting, one may instead wish to learn useful information from heterogeneous sources, or from real dynamical systems observations that are messy or incomplete. In this work, we learn general-purpose representations of PDEs from heterogeneous data by implementing joint embedding methods for self-supervised learning (SSL), a framework for unsupervised representation learning that has had notable success in computer vision. Our representation outperforms baseline approaches to invariant tasks, such as regressing the coefficients of a PDE, while also improving the time-stepping performance of neural solvers. We hope that our proposed methodology will prove useful in the eventual development of general-purpose foundation models for PDEs. Code: https://github.com/facebookresearch/SSLForPDEs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2307_05432 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Self-Supervised Learning with Lie Symmetries for Partial Differential Equations Mialon, Grégoire Garrido, Quentin Lawrence, Hannah Rehman, Danyal LeCun, Yann Kiani, Bobak T. Machine Learning Numerical Analysis Machine learning for differential equations paves the way for computationally efficient alternatives to numerical solvers, with potentially broad impacts in science and engineering. Though current algorithms typically require simulated training data tailored to a given setting, one may instead wish to learn useful information from heterogeneous sources, or from real dynamical systems observations that are messy or incomplete. In this work, we learn general-purpose representations of PDEs from heterogeneous data by implementing joint embedding methods for self-supervised learning (SSL), a framework for unsupervised representation learning that has had notable success in computer vision. Our representation outperforms baseline approaches to invariant tasks, such as regressing the coefficients of a PDE, while also improving the time-stepping performance of neural solvers. We hope that our proposed methodology will prove useful in the eventual development of general-purpose foundation models for PDEs. Code: https://github.com/facebookresearch/SSLForPDEs. |
| title | Self-Supervised Learning with Lie Symmetries for Partial Differential Equations |
| topic | Machine Learning Numerical Analysis |
| url | https://arxiv.org/abs/2307.05432 |