Benchmarking Abstractive Summarisation: A Dataset of Human-authored Summaries of Norwegian News Articles
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909455617097728 |
|---|---|
| author | Touileb, Samia Mikhailov, Vladislav Kroka, Marie Øvrelid, Lilja Velldal, Erik |
| author_facet | Touileb, Samia Mikhailov, Vladislav Kroka, Marie Øvrelid, Lilja Velldal, Erik |
| contents | We introduce a dataset of high-quality human-authored summaries of news articles in Norwegian. The dataset is intended for benchmarking the abstractive summarisation capabilities of generative language models. Each document in the dataset is provided with three different candidate gold-standard summaries written by native Norwegian speakers, and all summaries are provided in both of the written variants of Norwegian -- Bokmål and Nynorsk. The paper describes details on the data creation effort as well as an evaluation of existing open LLMs for Norwegian on the dataset. We also provide insights from a manual human evaluation, comparing human-authored to model-generated summaries. Our results indicate that the dataset provides a challenging LLM benchmark for Norwegian summarisation capabilities |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_07718 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Benchmarking Abstractive Summarisation: A Dataset of Human-authored Summaries of Norwegian News Articles Touileb, Samia Mikhailov, Vladislav Kroka, Marie Øvrelid, Lilja Velldal, Erik Computation and Language We introduce a dataset of high-quality human-authored summaries of news articles in Norwegian. The dataset is intended for benchmarking the abstractive summarisation capabilities of generative language models. Each document in the dataset is provided with three different candidate gold-standard summaries written by native Norwegian speakers, and all summaries are provided in both of the written variants of Norwegian -- Bokmål and Nynorsk. The paper describes details on the data creation effort as well as an evaluation of existing open LLMs for Norwegian on the dataset. We also provide insights from a manual human evaluation, comparing human-authored to model-generated summaries. Our results indicate that the dataset provides a challenging LLM benchmark for Norwegian summarisation capabilities |
| title | Benchmarking Abstractive Summarisation: A Dataset of Human-authored Summaries of Norwegian News Articles |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2501.07718 |