CLLMate: A Multimodal Benchmark for Weather and Climate Events Forecasting
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912232667873280 |
|---|---|
| author | Li, Haobo Wang, Zhaowei Wang, Jiachen Wang, Yueya Lau, Alexis Kai Hon Qu, Huamin |
| author_facet | Li, Haobo Wang, Zhaowei Wang, Jiachen Wang, Yueya Lau, Alexis Kai Hon Qu, Huamin |
| contents | Forecasting weather and climate events is crucial for making appropriate measures to mitigate environmental hazards and minimize losses. However, existing environmental forecasting research focuses narrowly on predicting numerical meteorological variables (e.g., temperature), neglecting the translation of these variables into actionable textual narratives of events and their consequences. To bridge this gap, we proposed Weather and Climate Event Forecasting (WCEF), a new task that leverages numerical meteorological raster data and textual event data to predict weather and climate events. This task is challenging to accomplish due to difficulties in aligning multimodal data and the lack of supervised datasets. To address these challenges, we present CLLMate, the first multimodal dataset for WCEF, using 26,156 environmental news articles aligned with ERA5 reanalysis data. We systematically benchmark 23 existing MLLMs on CLLMate, including closed-source, open-source, and our fine-tuned models. Our experiments reveal the advantages and limitations of existing MLLMs and the value of CLLMate for the training and benchmarking of the WCEF task. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_19058 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | CLLMate: A Multimodal Benchmark for Weather and Climate Events Forecasting Li, Haobo Wang, Zhaowei Wang, Jiachen Wang, Yueya Lau, Alexis Kai Hon Qu, Huamin Machine Learning Artificial Intelligence Computation and Language Atmospheric and Oceanic Physics Forecasting weather and climate events is crucial for making appropriate measures to mitigate environmental hazards and minimize losses. However, existing environmental forecasting research focuses narrowly on predicting numerical meteorological variables (e.g., temperature), neglecting the translation of these variables into actionable textual narratives of events and their consequences. To bridge this gap, we proposed Weather and Climate Event Forecasting (WCEF), a new task that leverages numerical meteorological raster data and textual event data to predict weather and climate events. This task is challenging to accomplish due to difficulties in aligning multimodal data and the lack of supervised datasets. To address these challenges, we present CLLMate, the first multimodal dataset for WCEF, using 26,156 environmental news articles aligned with ERA5 reanalysis data. We systematically benchmark 23 existing MLLMs on CLLMate, including closed-source, open-source, and our fine-tuned models. Our experiments reveal the advantages and limitations of existing MLLMs and the value of CLLMate for the training and benchmarking of the WCEF task. |
| title | CLLMate: A Multimodal Benchmark for Weather and Climate Events Forecasting |
| topic | Machine Learning Artificial Intelligence Computation and Language Atmospheric and Oceanic Physics |
| url | https://arxiv.org/abs/2409.19058 |