TURNA: A Turkish Encoder-Decoder Language Model for Enhanced Understanding and Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911364417585152 |
|---|---|
| author | Uludoğan, Gökçe Balal, Zeynep Yirmibeşoğlu Akkurt, Furkan Türker, Melikşah Güngör, Onur Üsküdarlı, Susan |
| author_facet | Uludoğan, Gökçe Balal, Zeynep Yirmibeşoğlu Akkurt, Furkan Türker, Melikşah Güngör, Onur Üsküdarlı, Susan |
| contents | The recent advances in natural language processing have predominantly favored well-resourced English-centric models, resulting in a significant gap with low-resource languages. In this work, we introduce the language model TURNA, which is developed for the low-resource language Turkish and is capable of both natural language understanding and generation tasks. TURNA is pretrained with an encoder-decoder architecture based on the unified framework UL2 with a diverse corpus that we specifically curated for this purpose. We evaluated TURNA with three generation tasks and five understanding tasks for Turkish. The results show that TURNA outperforms several multilingual models in both understanding and generation tasks, and competes with monolingual Turkish models in understanding tasks. TURNA is made available at https://huggingface.co/boun-tabi-LMG/TURNA . |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_14373 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | TURNA: A Turkish Encoder-Decoder Language Model for Enhanced Understanding and Generation Uludoğan, Gökçe Balal, Zeynep Yirmibeşoğlu Akkurt, Furkan Türker, Melikşah Güngör, Onur Üsküdarlı, Susan Computation and Language Artificial Intelligence Machine Learning The recent advances in natural language processing have predominantly favored well-resourced English-centric models, resulting in a significant gap with low-resource languages. In this work, we introduce the language model TURNA, which is developed for the low-resource language Turkish and is capable of both natural language understanding and generation tasks. TURNA is pretrained with an encoder-decoder architecture based on the unified framework UL2 with a diverse corpus that we specifically curated for this purpose. We evaluated TURNA with three generation tasks and five understanding tasks for Turkish. The results show that TURNA outperforms several multilingual models in both understanding and generation tasks, and competes with monolingual Turkish models in understanding tasks. TURNA is made available at https://huggingface.co/boun-tabi-LMG/TURNA . |
| title | TURNA: A Turkish Encoder-Decoder Language Model for Enhanced Understanding and Generation |
| topic | Computation and Language Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2401.14373 |