Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2505.18653 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912392507555840 |
|---|---|
| author | Kurfalı, Murathan Zahra, Shorouq Nivre, Joakim Messori, Gabriele |
| author_facet | Kurfalı, Murathan Zahra, Shorouq Nivre, Joakim Messori, Gabriele |
| contents | Climate-Eval is a comprehensive benchmark designed to evaluate natural language processing models across a broad range of tasks related to climate change. Climate-Eval aggregates existing datasets along with a newly developed news classification dataset, created specifically for this release. This results in a benchmark of 25 tasks based on 13 datasets, covering key aspects of climate discourse, including text classification, question answering, and information extraction. Our benchmark provides a standardized evaluation suite for systematically assessing the performance of large language models (LLMs) on these tasks. Additionally, we conduct an extensive evaluation of open-source LLMs (ranging from 2B to 70B parameters) in both zero-shot and few-shot settings, analyzing their strengths and limitations in the domain of climate change. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_18653 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Climate-Eval: A Comprehensive Benchmark for NLP Tasks Related to Climate Change Kurfalı, Murathan Zahra, Shorouq Nivre, Joakim Messori, Gabriele Computation and Language Climate-Eval is a comprehensive benchmark designed to evaluate natural language processing models across a broad range of tasks related to climate change. Climate-Eval aggregates existing datasets along with a newly developed news classification dataset, created specifically for this release. This results in a benchmark of 25 tasks based on 13 datasets, covering key aspects of climate discourse, including text classification, question answering, and information extraction. Our benchmark provides a standardized evaluation suite for systematically assessing the performance of large language models (LLMs) on these tasks. Additionally, we conduct an extensive evaluation of open-source LLMs (ranging from 2B to 70B parameters) in both zero-shot and few-shot settings, analyzing their strengths and limitations in the domain of climate change. |
| title | Climate-Eval: A Comprehensive Benchmark for NLP Tasks Related to Climate Change |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2505.18653 |