Scalable Climate Data Analysis: Balancing Petascale Fidelity and Computational Cost
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915400237711360 |
|---|---|
| author | Panta, Aashish Gooch, Amy Scorzelli, Giorgio Taufer, Michela Pascucci, Valerio |
| author_facet | Panta, Aashish Gooch, Amy Scorzelli, Giorgio Taufer, Michela Pascucci, Valerio |
| contents | The growing resolution and volume of climate data from remote sensing and simulations pose significant storage, processing, and computational challenges. Traditional compression or subsampling methods often compromise data fidelity, limiting scientific insights. We introduce a scalable ecosystem that integrates hierarchical multiresolution data management, intelligent transmission, and ML-assisted reconstruction to balance accuracy and efficiency. Our approach reduces storage and computational costs by 99\%, lowering expenses from \$100,000 to \$24 while maintaining a Root Mean Square (RMS) error of 1.46 degrees Celsius. Our experimental results confirm that even with significant data reduction, essential features required for accurate climate analysis are preserved. Validated on petascale NASA climate datasets, this solution enables cost-effective, high-fidelity climate analysis for research and decision-making. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_08006 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Scalable Climate Data Analysis: Balancing Petascale Fidelity and Computational Cost Panta, Aashish Gooch, Amy Scorzelli, Giorgio Taufer, Michela Pascucci, Valerio Atmospheric and Oceanic Physics Human-Computer Interaction The growing resolution and volume of climate data from remote sensing and simulations pose significant storage, processing, and computational challenges. Traditional compression or subsampling methods often compromise data fidelity, limiting scientific insights. We introduce a scalable ecosystem that integrates hierarchical multiresolution data management, intelligent transmission, and ML-assisted reconstruction to balance accuracy and efficiency. Our approach reduces storage and computational costs by 99\%, lowering expenses from \$100,000 to \$24 while maintaining a Root Mean Square (RMS) error of 1.46 degrees Celsius. Our experimental results confirm that even with significant data reduction, essential features required for accurate climate analysis are preserved. Validated on petascale NASA climate datasets, this solution enables cost-effective, high-fidelity climate analysis for research and decision-making. |
| title | Scalable Climate Data Analysis: Balancing Petascale Fidelity and Computational Cost |
| topic | Atmospheric and Oceanic Physics Human-Computer Interaction |
| url | https://arxiv.org/abs/2507.08006 |