IT5: Text-to-text Pretraining for Italian Language Understanding and Generation
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2022
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916250646478848 |
|---|---|
| author | Sarti, Gabriele Nissim, Malvina |
| author_facet | Sarti, Gabriele Nissim, Malvina |
| contents | We introduce IT5, the first family of encoder-decoder transformer models pretrained specifically on Italian. We document and perform a thorough cleaning procedure for a large Italian corpus and use it to pretrain four IT5 model sizes. We then introduce the ItaGen benchmark, which includes a broad range of natural language understanding and generation tasks for Italian, and use it to evaluate the performance of IT5 models and multilingual baselines. We find monolingual IT5 models to provide the best scale-to-performance ratio across tested models, consistently outperforming their multilingual counterparts and setting a new state-of-the-art for Italian language generation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2203_03759 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | IT5: Text-to-text Pretraining for Italian Language Understanding and Generation Sarti, Gabriele Nissim, Malvina Computation and Language We introduce IT5, the first family of encoder-decoder transformer models pretrained specifically on Italian. We document and perform a thorough cleaning procedure for a large Italian corpus and use it to pretrain four IT5 model sizes. We then introduce the ItaGen benchmark, which includes a broad range of natural language understanding and generation tasks for Italian, and use it to evaluate the performance of IT5 models and multilingual baselines. We find monolingual IT5 models to provide the best scale-to-performance ratio across tested models, consistently outperforming their multilingual counterparts and setting a new state-of-the-art for Italian language generation. |
| title | IT5: Text-to-text Pretraining for Italian Language Understanding and Generation |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2203.03759 |