ORBIT: Oak Ridge Base Foundation Model for Earth System Predictability
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909290675044352 |
|---|---|
| author | Wang, Xiao Liu, Siyan Tsaris, Aristeidis Choi, Jong-Youl Aji, Ashwin Fan, Ming Zhang, Wei Yin, Junqi Ashfaq, Moetasim Lu, Dan Balaprakash, Prasanna |
| author_facet | Wang, Xiao Liu, Siyan Tsaris, Aristeidis Choi, Jong-Youl Aji, Ashwin Fan, Ming Zhang, Wei Yin, Junqi Ashfaq, Moetasim Lu, Dan Balaprakash, Prasanna |
| contents | Earth system predictability is challenged by the complexity of environmental dynamics and the multitude of variables involved. Current AI foundation models, although advanced by leveraging large and heterogeneous data, are often constrained by their size and data integration, limiting their effectiveness in addressing the full range of Earth system prediction challenges. To overcome these limitations, we introduce the Oak Ridge Base Foundation Model for Earth System Predictability (ORBIT), an advanced vision transformer model that scales up to 113 billion parameters using a novel hybrid tensor-data orthogonal parallelism technique. As the largest model of its kind, ORBIT surpasses the current climate AI foundation model size by a thousandfold. Performance scaling tests conducted on the Frontier supercomputer have demonstrated that ORBIT achieves 684 petaFLOPS to 1.6 exaFLOPS sustained throughput, with scaling efficiency maintained at 41% to 85% across 49,152 AMD GPUs. These breakthroughs establish new advances in AI-driven climate modeling and demonstrate promise to significantly improve the Earth system predictability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_14712 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ORBIT: Oak Ridge Base Foundation Model for Earth System Predictability Wang, Xiao Liu, Siyan Tsaris, Aristeidis Choi, Jong-Youl Aji, Ashwin Fan, Ming Zhang, Wei Yin, Junqi Ashfaq, Moetasim Lu, Dan Balaprakash, Prasanna Atmospheric and Oceanic Physics Artificial Intelligence Distributed, Parallel, and Cluster Computing Image and Video Processing Geophysics Earth system predictability is challenged by the complexity of environmental dynamics and the multitude of variables involved. Current AI foundation models, although advanced by leveraging large and heterogeneous data, are often constrained by their size and data integration, limiting their effectiveness in addressing the full range of Earth system prediction challenges. To overcome these limitations, we introduce the Oak Ridge Base Foundation Model for Earth System Predictability (ORBIT), an advanced vision transformer model that scales up to 113 billion parameters using a novel hybrid tensor-data orthogonal parallelism technique. As the largest model of its kind, ORBIT surpasses the current climate AI foundation model size by a thousandfold. Performance scaling tests conducted on the Frontier supercomputer have demonstrated that ORBIT achieves 684 petaFLOPS to 1.6 exaFLOPS sustained throughput, with scaling efficiency maintained at 41% to 85% across 49,152 AMD GPUs. These breakthroughs establish new advances in AI-driven climate modeling and demonstrate promise to significantly improve the Earth system predictability. |
| title | ORBIT: Oak Ridge Base Foundation Model for Earth System Predictability |
| topic | Atmospheric and Oceanic Physics Artificial Intelligence Distributed, Parallel, and Cluster Computing Image and Video Processing Geophysics |
| url | https://arxiv.org/abs/2404.14712 |