Towards Reasoning for PDE Foundation Models: A Reward-Model-Driven Inference-Time-Scaling Algorithm
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917219682746368 |
|---|---|
| author | Mansingh, Siddharth Amarel, James Arnab, Ragib Mohan, Arvind Singh, Kamaljeet Kunde, Gerd J. Hengartner, Nicolas Migliori, Benjamin Casleton, Emily Debardeleben, Nathan A. Biswas, Ayan Oyen, Diane Lawrence, Earl |
| author_facet | Mansingh, Siddharth Amarel, James Arnab, Ragib Mohan, Arvind Singh, Kamaljeet Kunde, Gerd J. Hengartner, Nicolas Migliori, Benjamin Casleton, Emily Debardeleben, Nathan A. Biswas, Ayan Oyen, Diane Lawrence, Earl |
| contents | Partial Differential Equations (PDEs) are the bedrock for modern computational sciences and engineering, and inherently computationally expensive. While PDE foundation models have shown much promise for simulating such complex spatio-temporal phenomena, existing models remain constrained by the pretraining datasets and struggle with auto-regressive rollout performance, especially in out-of-distribution (OOD) cases. Furthermore, they have significant compute and training data requirements which hamper their use in many critical applications. Inspired by recent advances in ``thinking" strategies used in large language models (LLMs), we introduce the first test-time computing (TTC) strategy for PDEs that utilizes computational resources during inference to achieve more accurate predictions with fewer training samples and smaller models. We accomplish this with two types of reward models that evaluate predictions of a stochastic based model for spatio-temporal consistency. We demonstrate this method on compressible Euler-equation simulations from the PDEGym benchmark and show that TTC captures improved predictions relative to standard non-adaptive auto-regressive inference. This TTC framework marks a foundational step towards more advanced reasoning algorithms or PDE modeling, inluding building reinforcement-learning-based approaches, potentially transforming computational workflows in physics and engineering. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_02846 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards Reasoning for PDE Foundation Models: A Reward-Model-Driven Inference-Time-Scaling Algorithm Mansingh, Siddharth Amarel, James Arnab, Ragib Mohan, Arvind Singh, Kamaljeet Kunde, Gerd J. Hengartner, Nicolas Migliori, Benjamin Casleton, Emily Debardeleben, Nathan A. Biswas, Ayan Oyen, Diane Lawrence, Earl Machine Learning Computational Physics Partial Differential Equations (PDEs) are the bedrock for modern computational sciences and engineering, and inherently computationally expensive. While PDE foundation models have shown much promise for simulating such complex spatio-temporal phenomena, existing models remain constrained by the pretraining datasets and struggle with auto-regressive rollout performance, especially in out-of-distribution (OOD) cases. Furthermore, they have significant compute and training data requirements which hamper their use in many critical applications. Inspired by recent advances in ``thinking" strategies used in large language models (LLMs), we introduce the first test-time computing (TTC) strategy for PDEs that utilizes computational resources during inference to achieve more accurate predictions with fewer training samples and smaller models. We accomplish this with two types of reward models that evaluate predictions of a stochastic based model for spatio-temporal consistency. We demonstrate this method on compressible Euler-equation simulations from the PDEGym benchmark and show that TTC captures improved predictions relative to standard non-adaptive auto-regressive inference. This TTC framework marks a foundational step towards more advanced reasoning algorithms or PDE modeling, inluding building reinforcement-learning-based approaches, potentially transforming computational workflows in physics and engineering. |
| title | Towards Reasoning for PDE Foundation Models: A Reward-Model-Driven Inference-Time-Scaling Algorithm |
| topic | Machine Learning Computational Physics |
| url | https://arxiv.org/abs/2509.02846 |