HeSum: a Novel Dataset for Abstractive Text Summarization in Hebrew
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916837742084096 |
|---|---|
| author | Paz-Argaman, Tzuf Mondshine, Itai Mordechai, Asaf Achi Tsarfaty, Reut |
| author_facet | Paz-Argaman, Tzuf Mondshine, Itai Mordechai, Asaf Achi Tsarfaty, Reut |
| contents | While large language models (LLMs) excel in various natural language tasks in English, their performance in lower-resourced languages like Hebrew, especially for generative tasks such as abstractive summarization, remains unclear. The high morphological richness in Hebrew adds further challenges due to the ambiguity in sentence comprehension and the complexities in meaning construction. In this paper, we address this resource and evaluation gap by introducing HeSum, a novel benchmark specifically designed for abstractive text summarization in Modern Hebrew. HeSum consists of 10,000 article-summary pairs sourced from Hebrew news websites written by professionals. Linguistic analysis confirms HeSum's high abstractness and unique morphological challenges. We show that HeSum presents distinct difficulties for contemporary state-of-the-art LLMs, establishing it as a valuable testbed for generative language technology in Hebrew, and MRLs generative challenges in general. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_03897 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | HeSum: a Novel Dataset for Abstractive Text Summarization in Hebrew Paz-Argaman, Tzuf Mondshine, Itai Mordechai, Asaf Achi Tsarfaty, Reut Computation and Language Artificial Intelligence While large language models (LLMs) excel in various natural language tasks in English, their performance in lower-resourced languages like Hebrew, especially for generative tasks such as abstractive summarization, remains unclear. The high morphological richness in Hebrew adds further challenges due to the ambiguity in sentence comprehension and the complexities in meaning construction. In this paper, we address this resource and evaluation gap by introducing HeSum, a novel benchmark specifically designed for abstractive text summarization in Modern Hebrew. HeSum consists of 10,000 article-summary pairs sourced from Hebrew news websites written by professionals. Linguistic analysis confirms HeSum's high abstractness and unique morphological challenges. We show that HeSum presents distinct difficulties for contemporary state-of-the-art LLMs, establishing it as a valuable testbed for generative language technology in Hebrew, and MRLs generative challenges in general. |
| title | HeSum: a Novel Dataset for Abstractive Text Summarization in Hebrew |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2406.03897 |