Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2604.06567 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913145985957888 |
|---|---|
| author | Pallotta, Giuliana Duan, Shiheng Bonfils, Céline Lee, Jiwoo Goodnight, Seth Ullrich, Paul |
| author_facet | Pallotta, Giuliana Duan, Shiheng Bonfils, Céline Lee, Jiwoo Goodnight, Seth Ullrich, Paul |
| contents | In recent years, Deep-Learning Earth System Models (DL-ESMs) have emerged as promising, computationally efficient complements to traditional Earth system models. Here, we present an evaluation framework for testing DL-ESMs from a climate-model-development perspective using standardized diagnostics from the PCMDI Metrics Package (PMP). This framework allows DL-ESMs, including Ai2's ACE2 and Google's NeuralGCM, to be assessed with metrics that quantify their ability to reproduce climatology, major modes of variability, monsoon behavior, and precipitation variability relative to observational reference datasets and CMIP-class benchmarks. By evaluating DL-ESMs with tools commonly used for traditional models, we extend their assessment beyond short-range forecast skill and toward climate-relevant applications. The results identify encouraging strengths in several large-scale fields and modes of variability, while also highlighting persistent challenges in precipitation, tropical variability, and long-run stability for some model versions. This evaluation is a critical step toward building trust in DL-ESMs, guiding future model development, and clarifying their fit-for-purpose for Earth system science applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_06567 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | A PMP-inspired Evaluation Framework for Assessing Deep-Learning Earth System Models Pallotta, Giuliana Duan, Shiheng Bonfils, Céline Lee, Jiwoo Goodnight, Seth Ullrich, Paul Atmospheric and Oceanic Physics In recent years, Deep-Learning Earth System Models (DL-ESMs) have emerged as promising, computationally efficient complements to traditional Earth system models. Here, we present an evaluation framework for testing DL-ESMs from a climate-model-development perspective using standardized diagnostics from the PCMDI Metrics Package (PMP). This framework allows DL-ESMs, including Ai2's ACE2 and Google's NeuralGCM, to be assessed with metrics that quantify their ability to reproduce climatology, major modes of variability, monsoon behavior, and precipitation variability relative to observational reference datasets and CMIP-class benchmarks. By evaluating DL-ESMs with tools commonly used for traditional models, we extend their assessment beyond short-range forecast skill and toward climate-relevant applications. The results identify encouraging strengths in several large-scale fields and modes of variability, while also highlighting persistent challenges in precipitation, tropical variability, and long-run stability for some model versions. This evaluation is a critical step toward building trust in DL-ESMs, guiding future model development, and clarifying their fit-for-purpose for Earth system science applications. |
| title | A PMP-inspired Evaluation Framework for Assessing Deep-Learning Earth System Models |
| topic | Atmospheric and Oceanic Physics |
| url | https://arxiv.org/abs/2604.06567 |