Event-based evaluation of abstractive news summarization
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916821003665408 |
|---|---|
| author | You, Huiling Touileb, Samia Velldal, Erik Øvrelid, Lilja |
| author_facet | You, Huiling Touileb, Samia Velldal, Erik Øvrelid, Lilja |
| contents | An abstractive summary of a news article contains its most important information in a condensed version. The evaluation of automatically generated summaries by generative language models relies heavily on human-authored summaries as gold references, by calculating overlapping units or similarity scores. News articles report events, and ideally so should the summaries. In this work, we propose to evaluate the quality of abstractive summaries by calculating overlapping events between generated summaries, reference summaries, and the original news articles. We experiment on a richly annotated Norwegian dataset comprising both events annotations and summaries authored by expert human annotators. Our approach provides more insight into the event information contained in the summaries. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_01160 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Event-based evaluation of abstractive news summarization You, Huiling Touileb, Samia Velldal, Erik Øvrelid, Lilja Computation and Language An abstractive summary of a news article contains its most important information in a condensed version. The evaluation of automatically generated summaries by generative language models relies heavily on human-authored summaries as gold references, by calculating overlapping units or similarity scores. News articles report events, and ideally so should the summaries. In this work, we propose to evaluate the quality of abstractive summaries by calculating overlapping events between generated summaries, reference summaries, and the original news articles. We experiment on a richly annotated Norwegian dataset comprising both events annotations and summaries authored by expert human annotators. Our approach provides more insight into the event information contained in the summaries. |
| title | Event-based evaluation of abstractive news summarization |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2507.01160 |