From News to Summaries: Building a Hungarian Corpus for Extractive and Abstractive Summarization
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909167257649152 |
|---|---|
| author | Barta, Botond Lakatos, Dorina Nagy, Attila Nyist, Milán Konor Ács, Judit |
| author_facet | Barta, Botond Lakatos, Dorina Nagy, Attila Nyist, Milán Konor Ács, Judit |
| contents | Training summarization models requires substantial amounts of training data. However for less resourceful languages like Hungarian, openly available models and datasets are notably scarce. To address this gap our paper introduces HunSum-2 an open-source Hungarian corpus suitable for training abstractive and extractive summarization models. The dataset is assembled from segments of the Common Crawl corpus undergoing thorough cleaning, preprocessing and deduplication. In addition to abstractive summarization we generate sentence-level labels for extractive summarization using sentence similarity. We train baseline models for both extractive and abstractive summarization using the collected dataset. To demonstrate the effectiveness of the trained models, we perform both quantitative and qualitative evaluation. Our dataset, models and code are publicly available, encouraging replication, further research, and real-world applications across various domains. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_03555 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | From News to Summaries: Building a Hungarian Corpus for Extractive and Abstractive Summarization Barta, Botond Lakatos, Dorina Nagy, Attila Nyist, Milán Konor Ács, Judit Computation and Language Training summarization models requires substantial amounts of training data. However for less resourceful languages like Hungarian, openly available models and datasets are notably scarce. To address this gap our paper introduces HunSum-2 an open-source Hungarian corpus suitable for training abstractive and extractive summarization models. The dataset is assembled from segments of the Common Crawl corpus undergoing thorough cleaning, preprocessing and deduplication. In addition to abstractive summarization we generate sentence-level labels for extractive summarization using sentence similarity. We train baseline models for both extractive and abstractive summarization using the collected dataset. To demonstrate the effectiveness of the trained models, we perform both quantitative and qualitative evaluation. Our dataset, models and code are publicly available, encouraging replication, further research, and real-world applications across various domains. |
| title | From News to Summaries: Building a Hungarian Corpus for Extractive and Abstractive Summarization |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2404.03555 |