GeoGrid-Bench: Can Foundation Models Understand Multimodal Gridded Geo-Spatial Data?
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909621847851008 |
|---|---|
| author | Jiang, Bowen Xie, Yangxinyu Wang, Xiaomeng He, Jiashu Bergerson, Joshua Hutchison, John K Branham, Jordan Taylor, Camillo J Mallick, Tanwi |
| author_facet | Jiang, Bowen Xie, Yangxinyu Wang, Xiaomeng He, Jiashu Bergerson, Joshua Hutchison, John K Branham, Jordan Taylor, Camillo J Mallick, Tanwi |
| contents | We present GeoGrid-Bench, a benchmark designed to evaluate the ability of foundation models to understand geo-spatial data in the grid structure. Geo-spatial datasets pose distinct challenges due to their dense numerical values, strong spatial and temporal dependencies, and unique multimodal representations including tabular data, heatmaps, and geographic visualizations. To assess how foundation models can support scientific research in this domain, GeoGrid-Bench features large-scale, real-world data covering 16 climate variables across 150 locations and extended time frames. The benchmark includes approximately 3,200 question-answer pairs, systematically generated from 8 domain expert-curated templates to reflect practical tasks encountered by human scientists. These range from basic queries at a single location and time to complex spatiotemporal comparisons across regions and periods. Our evaluation reveals that vision-language models perform best overall, and we provide a fine-grained analysis of the strengths and limitations of different foundation models in different geo-spatial tasks. This benchmark offers clearer insights into how foundation models can be effectively applied to geo-spatial data analysis and used to support scientific research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_10714 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | GeoGrid-Bench: Can Foundation Models Understand Multimodal Gridded Geo-Spatial Data? Jiang, Bowen Xie, Yangxinyu Wang, Xiaomeng He, Jiashu Bergerson, Joshua Hutchison, John K Branham, Jordan Taylor, Camillo J Mallick, Tanwi Computation and Language We present GeoGrid-Bench, a benchmark designed to evaluate the ability of foundation models to understand geo-spatial data in the grid structure. Geo-spatial datasets pose distinct challenges due to their dense numerical values, strong spatial and temporal dependencies, and unique multimodal representations including tabular data, heatmaps, and geographic visualizations. To assess how foundation models can support scientific research in this domain, GeoGrid-Bench features large-scale, real-world data covering 16 climate variables across 150 locations and extended time frames. The benchmark includes approximately 3,200 question-answer pairs, systematically generated from 8 domain expert-curated templates to reflect practical tasks encountered by human scientists. These range from basic queries at a single location and time to complex spatiotemporal comparisons across regions and periods. Our evaluation reveals that vision-language models perform best overall, and we provide a fine-grained analysis of the strengths and limitations of different foundation models in different geo-spatial tasks. This benchmark offers clearer insights into how foundation models can be effectively applied to geo-spatial data analysis and used to support scientific research. |
| title | GeoGrid-Bench: Can Foundation Models Understand Multimodal Gridded Geo-Spatial Data? |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2505.10714 |