OmniArch: Building Foundation Model For Scientific Computing
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909626278084608 |
|---|---|
| author | Chen, Tianyu Zhou, Haoyi Li, Ying Wang, Hao Gao, Chonghan Shi, Rongye Zhang, Shanghang Li, Jianxin |
| author_facet | Chen, Tianyu Zhou, Haoyi Li, Ying Wang, Hao Gao, Chonghan Shi, Rongye Zhang, Shanghang Li, Jianxin |
| contents | Foundation models have revolutionized language modeling, while whether this success is replicated in scientific computing remains unexplored. We present OmniArch, the first prototype aiming at solving multi-scale and multi-physics scientific computing problems with physical alignment. We addressed all three challenges with one unified architecture. Its pre-training stage contains a Fourier Encoder-decoder fading out the disharmony across separated dimensions and a Transformer backbone integrating quantities through temporal dynamics, and the novel PDE-Aligner performs physics-informed fine-tuning under flexible conditions. As far as we know, we first conduct 1D-2D-3D united pre-training on the PDEBench, and it sets not only new performance benchmarks for 1D, 2D, and 3D PDEs but also demonstrates exceptional adaptability to new physics via in-context and zero-shot learning approaches, which supports realistic engineering applications and foresight physics discovery. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_16014 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | OmniArch: Building Foundation Model For Scientific Computing Chen, Tianyu Zhou, Haoyi Li, Ying Wang, Hao Gao, Chonghan Shi, Rongye Zhang, Shanghang Li, Jianxin Machine Learning Artificial Intelligence Foundation models have revolutionized language modeling, while whether this success is replicated in scientific computing remains unexplored. We present OmniArch, the first prototype aiming at solving multi-scale and multi-physics scientific computing problems with physical alignment. We addressed all three challenges with one unified architecture. Its pre-training stage contains a Fourier Encoder-decoder fading out the disharmony across separated dimensions and a Transformer backbone integrating quantities through temporal dynamics, and the novel PDE-Aligner performs physics-informed fine-tuning under flexible conditions. As far as we know, we first conduct 1D-2D-3D united pre-training on the PDEBench, and it sets not only new performance benchmarks for 1D, 2D, and 3D PDEs but also demonstrates exceptional adaptability to new physics via in-context and zero-shot learning approaches, which supports realistic engineering applications and foresight physics discovery. |
| title | OmniArch: Building Foundation Model For Scientific Computing |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2402.16014 |