Two Approaches to Diachronic Normalization of Polish Texts
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909090869936128 |
|---|---|
| author | Dudzic, Kacper Graliński, Filip Jassem, Krzysztof Kubis, Marek Wierzchoń, Piotr |
| author_facet | Dudzic, Kacper Graliński, Filip Jassem, Krzysztof Kubis, Marek Wierzchoń, Piotr |
| contents | This paper discusses two approaches to the diachronic normalization of Polish texts: a rule-based solution that relies on a set of handcrafted patterns, and a neural normalization model based on the text-to-text transfer transformer architecture. The training and evaluation data prepared for the task are discussed in detail, along with experiments conducted to compare the proposed normalization solutions. A quantitative and qualitative analysis is made. It is shown that at the current stage of inquiry into the problem, the rule-based solution outperforms the neural one on 3 out of 4 variants of the prepared dataset, although in practice both approaches have distinct advantages and disadvantages. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_01300 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Two Approaches to Diachronic Normalization of Polish Texts Dudzic, Kacper Graliński, Filip Jassem, Krzysztof Kubis, Marek Wierzchoń, Piotr Computation and Language This paper discusses two approaches to the diachronic normalization of Polish texts: a rule-based solution that relies on a set of handcrafted patterns, and a neural normalization model based on the text-to-text transfer transformer architecture. The training and evaluation data prepared for the task are discussed in detail, along with experiments conducted to compare the proposed normalization solutions. A quantitative and qualitative analysis is made. It is shown that at the current stage of inquiry into the problem, the rule-based solution outperforms the neural one on 3 out of 4 variants of the prepared dataset, although in practice both approaches have distinct advantages and disadvantages. |
| title | Two Approaches to Diachronic Normalization of Polish Texts |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2402.01300 |