Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.21670 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908797329473536 |
|---|---|
| author | Rautela, Mahindra Singh Most, Alexander Mansingh, Siddharth Love, Bradley C. Scheinker, Alexander Oyen, Diane Debardeleben, Nathan Lawrence, Earl Biswas, Ayan |
| author_facet | Rautela, Mahindra Singh Most, Alexander Mansingh, Siddharth Love, Bradley C. Scheinker, Alexander Oyen, Diane Debardeleben, Nathan Lawrence, Earl Biswas, Ayan |
| contents | We introduce MORPH, a modality-agnostic, autoregressive foundation model for partial differential equations (PDEs). MORPH is built on a convolutional vision transformer backbone that seamlessly handles heterogeneous spatiotemporal datasets of varying data modality (1D--3D) at different resolutions, and multiple fields with mixed scalar and vector components. The architecture combines (i) component-wise convolution, which jointly processes scalar and vector channels to capture local interactions, (ii) inter-field cross-attention, which models and selectively propagates information between different physical fields, (iii) axial attentions, which factorize full spatiotemporal self-attention along individual spatial and temporal axes to reduce computational burden while retaining expressivity. We pretrain multiple model variants on a diverse collection of heterogeneous PDE datasets and evaluate transfer to a range of downstream prediction tasks. Using both full-model fine-tuning and parameter-efficient low-rank adapters, MORPH outperforms models trained from scratch. Across extensive evaluations, MORPH matches or surpasses strong baselines and recent state-of-the-art models. Collectively, these capabilities present a flexible and powerful backbone for learning from the heterogeneous and multimodal nature of scientific observations, charting a path toward scalable and data-efficient scientific machine learning. The source code, datasets, and models are publicly available at https://github.com/lanl/MORPH. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_21670 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MORPH: PDE Foundation Models with Arbitrary Data Modality Rautela, Mahindra Singh Most, Alexander Mansingh, Siddharth Love, Bradley C. Scheinker, Alexander Oyen, Diane Debardeleben, Nathan Lawrence, Earl Biswas, Ayan Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning Computational Physics We introduce MORPH, a modality-agnostic, autoregressive foundation model for partial differential equations (PDEs). MORPH is built on a convolutional vision transformer backbone that seamlessly handles heterogeneous spatiotemporal datasets of varying data modality (1D--3D) at different resolutions, and multiple fields with mixed scalar and vector components. The architecture combines (i) component-wise convolution, which jointly processes scalar and vector channels to capture local interactions, (ii) inter-field cross-attention, which models and selectively propagates information between different physical fields, (iii) axial attentions, which factorize full spatiotemporal self-attention along individual spatial and temporal axes to reduce computational burden while retaining expressivity. We pretrain multiple model variants on a diverse collection of heterogeneous PDE datasets and evaluate transfer to a range of downstream prediction tasks. Using both full-model fine-tuning and parameter-efficient low-rank adapters, MORPH outperforms models trained from scratch. Across extensive evaluations, MORPH matches or surpasses strong baselines and recent state-of-the-art models. Collectively, these capabilities present a flexible and powerful backbone for learning from the heterogeneous and multimodal nature of scientific observations, charting a path toward scalable and data-efficient scientific machine learning. The source code, datasets, and models are publicly available at https://github.com/lanl/MORPH. |
| title | MORPH: PDE Foundation Models with Arbitrary Data Modality |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning Computational Physics |
| url | https://arxiv.org/abs/2509.21670 |