Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2403.12553 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866912098922004480 |
|---|---|
| author | Rahman, Md Ashiqur George, Robert Joseph Elleithy, Mogab Leibovici, Daniel Li, Zongyi Bonev, Boris White, Colin Berner, Julius Yeh, Raymond A. Kossaifi, Jean Azizzadenesheli, Kamyar Anandkumar, Anima |
| author_facet | Rahman, Md Ashiqur George, Robert Joseph Elleithy, Mogab Leibovici, Daniel Li, Zongyi Bonev, Boris White, Colin Berner, Julius Yeh, Raymond A. Kossaifi, Jean Azizzadenesheli, Kamyar Anandkumar, Anima |
| contents | Existing neural operator architectures face challenges when solving multiphysics problems with coupled partial differential equations (PDEs) due to complex geometries, interactions between physical variables, and the limited amounts of high-resolution training data. To address these issues, we propose Codomain Attention Neural Operator (CoDA-NO), which tokenizes functions along the codomain or channel space, enabling self-supervised learning or pretraining of multiple PDE systems. Specifically, we extend positional encoding, self-attention, and normalization layers to function spaces. CoDA-NO can learn representations of different PDE systems with a single model. We evaluate CoDA-NO's potential as a backbone for learning multiphysics PDEs over multiple systems by considering few-shot learning settings. On complex downstream tasks with limited data, such as fluid flow simulations, fluid-structure interactions, and Rayleigh-Bénard convection, we found CoDA-NO to outperform existing methods by over 36%. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_12553 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Pretraining Codomain Attention Neural Operators for Solving Multiphysics PDEs Rahman, Md Ashiqur George, Robert Joseph Elleithy, Mogab Leibovici, Daniel Li, Zongyi Bonev, Boris White, Colin Berner, Julius Yeh, Raymond A. Kossaifi, Jean Azizzadenesheli, Kamyar Anandkumar, Anima Machine Learning Existing neural operator architectures face challenges when solving multiphysics problems with coupled partial differential equations (PDEs) due to complex geometries, interactions between physical variables, and the limited amounts of high-resolution training data. To address these issues, we propose Codomain Attention Neural Operator (CoDA-NO), which tokenizes functions along the codomain or channel space, enabling self-supervised learning or pretraining of multiple PDE systems. Specifically, we extend positional encoding, self-attention, and normalization layers to function spaces. CoDA-NO can learn representations of different PDE systems with a single model. We evaluate CoDA-NO's potential as a backbone for learning multiphysics PDEs over multiple systems by considering few-shot learning settings. On complex downstream tasks with limited data, such as fluid flow simulations, fluid-structure interactions, and Rayleigh-Bénard convection, we found CoDA-NO to outperform existing methods by over 36%. |
| title | Pretraining Codomain Attention Neural Operators for Solving Multiphysics PDEs |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2403.12553 |