Walrus: A Cross-Domain Foundation Model for Continuum Dynamics
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917093166809088 |
|---|---|
| author | McCabe, Michael Mukhopadhyay, Payel Marwah, Tanya Blancard, Bruno Regaldo-Saint Rozet, Francois Diaconu, Cristiana Meyer, Lucas Wong, Kaze W. K. Sotoudeh, Hadi Bietti, Alberto Espejo, Irina Fear, Rio Golkar, Siavash Hehir, Tom Hirashima, Keiya Krawezik, Geraud Lanusse, Francois Morel, Rudy Ohana, Ruben Parker, Liam Pettee, Mariel Shen, Jeff Cho, Kyunghyun Cranmer, Miles Ho, Shirley |
| author_facet | McCabe, Michael Mukhopadhyay, Payel Marwah, Tanya Blancard, Bruno Regaldo-Saint Rozet, Francois Diaconu, Cristiana Meyer, Lucas Wong, Kaze W. K. Sotoudeh, Hadi Bietti, Alberto Espejo, Irina Fear, Rio Golkar, Siavash Hehir, Tom Hirashima, Keiya Krawezik, Geraud Lanusse, Francois Morel, Rudy Ohana, Ruben Parker, Liam Pettee, Mariel Shen, Jeff Cho, Kyunghyun Cranmer, Miles Ho, Shirley |
| contents | Foundation models have transformed machine learning for language and vision, but achieving comparable impact in physical simulation remains a challenge. Data heterogeneity and unstable long-term dynamics inhibit learning from sufficiently diverse dynamics, while varying resolutions and dimensionalities challenge efficient training on modern hardware. Through empirical and theoretical analysis, we incorporate new approaches to mitigate these obstacles, including a harmonic-analysis-based stabilization method, load-balanced distributed 2D and 3D training strategies, and compute-adaptive tokenization. Using these tools, we develop Walrus, a transformer-based foundation model developed primarily for fluid-like continuum dynamics. Walrus is pretrained on nineteen diverse scenarios spanning astrophysics, geoscience, rheology, plasma physics, acoustics, and classical fluids. Experiments show that Walrus outperforms prior foundation models on both short and long term prediction horizons on downstream tasks and across the breadth of pretraining data, while ablation studies confirm the value of our contributions to forecast stability, training throughput, and transfer performance over conventional approaches. Code and weights are released for community use. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_15684 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Walrus: A Cross-Domain Foundation Model for Continuum Dynamics McCabe, Michael Mukhopadhyay, Payel Marwah, Tanya Blancard, Bruno Regaldo-Saint Rozet, Francois Diaconu, Cristiana Meyer, Lucas Wong, Kaze W. K. Sotoudeh, Hadi Bietti, Alberto Espejo, Irina Fear, Rio Golkar, Siavash Hehir, Tom Hirashima, Keiya Krawezik, Geraud Lanusse, Francois Morel, Rudy Ohana, Ruben Parker, Liam Pettee, Mariel Shen, Jeff Cho, Kyunghyun Cranmer, Miles Ho, Shirley Machine Learning Artificial Intelligence Computational Engineering, Finance, and Science Foundation models have transformed machine learning for language and vision, but achieving comparable impact in physical simulation remains a challenge. Data heterogeneity and unstable long-term dynamics inhibit learning from sufficiently diverse dynamics, while varying resolutions and dimensionalities challenge efficient training on modern hardware. Through empirical and theoretical analysis, we incorporate new approaches to mitigate these obstacles, including a harmonic-analysis-based stabilization method, load-balanced distributed 2D and 3D training strategies, and compute-adaptive tokenization. Using these tools, we develop Walrus, a transformer-based foundation model developed primarily for fluid-like continuum dynamics. Walrus is pretrained on nineteen diverse scenarios spanning astrophysics, geoscience, rheology, plasma physics, acoustics, and classical fluids. Experiments show that Walrus outperforms prior foundation models on both short and long term prediction horizons on downstream tasks and across the breadth of pretraining data, while ablation studies confirm the value of our contributions to forecast stability, training throughput, and transfer performance over conventional approaches. Code and weights are released for community use. |
| title | Walrus: A Cross-Domain Foundation Model for Continuum Dynamics |
| topic | Machine Learning Artificial Intelligence Computational Engineering, Finance, and Science |
| url | https://arxiv.org/abs/2511.15684 |