Forging GEMs: Advancing Greek NLP through Quality-Based Corpus Curation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914111036588032 |
|---|---|
| author | Apostolopoulou, Alexandra Kanaris, Konstantinos Koursaris, Athanasios Tsakalidis, Dimitris Domalis, George Livieris, Ioannis E. |
| author_facet | Apostolopoulou, Alexandra Kanaris, Konstantinos Koursaris, Athanasios Tsakalidis, Dimitris Domalis, George Livieris, Ioannis E. |
| contents | The advancement of natural language processing for morphologically rich and moderately-resourced languages like Modern Greek has been hindered by architectural stagnation, data scarcity, and limited context processing capabilities, particularly in specialized domains such as law. In this work, we propose the Greek Embedding Models (GEMs), a new family of transformer-based language models, specifically developed to address these limitations through architectural diversity and enhanced data curation. The proposed family of models are trained on several large-scale, meticulously curated corpora, encompassing both comprehensive general-domain datasets and specialized legal collections, addressing the persistent data scarcity that has impeded Greek language modeling advancement. The proposed quality-based corpus curation methodology incorporates extensive preprocessing pipelines, sophisticated deduplication strategies and targeted repetition of high-quality legal sub-corpora to enhance domain adaptation. The GEMs family comprises both established architectures (RoBERTa and Longformer) and advanced models not previously applied to Greek (ELECTRA, ConvBERT, and ModernBERT), providing comprehensive coverage of modern transformer designs. Additionally, we introduce the first bilingual Greek-English embedding models tailored for cross-lingual legal applications. Comprehensive evaluation across three core natural language understanding benchmarks demonstrates that the proposed GEM-RoBERTa and GEM-ConvBERT achieve statistically significant performance improvements over established state-of-the-art models, with accuracy gains of up to 3.6\% while conducted statistical analysis using Friedman Aligned-Ranks and Finner post-hoc tests confirms the superiority of our approach across multiple evaluation metrics. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_20002 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Forging GEMs: Advancing Greek NLP through Quality-Based Corpus Curation Apostolopoulou, Alexandra Kanaris, Konstantinos Koursaris, Athanasios Tsakalidis, Dimitris Domalis, George Livieris, Ioannis E. Computation and Language Artificial Intelligence 68T50, 68T07, 68U35 The advancement of natural language processing for morphologically rich and moderately-resourced languages like Modern Greek has been hindered by architectural stagnation, data scarcity, and limited context processing capabilities, particularly in specialized domains such as law. In this work, we propose the Greek Embedding Models (GEMs), a new family of transformer-based language models, specifically developed to address these limitations through architectural diversity and enhanced data curation. The proposed family of models are trained on several large-scale, meticulously curated corpora, encompassing both comprehensive general-domain datasets and specialized legal collections, addressing the persistent data scarcity that has impeded Greek language modeling advancement. The proposed quality-based corpus curation methodology incorporates extensive preprocessing pipelines, sophisticated deduplication strategies and targeted repetition of high-quality legal sub-corpora to enhance domain adaptation. The GEMs family comprises both established architectures (RoBERTa and Longformer) and advanced models not previously applied to Greek (ELECTRA, ConvBERT, and ModernBERT), providing comprehensive coverage of modern transformer designs. Additionally, we introduce the first bilingual Greek-English embedding models tailored for cross-lingual legal applications. Comprehensive evaluation across three core natural language understanding benchmarks demonstrates that the proposed GEM-RoBERTa and GEM-ConvBERT achieve statistically significant performance improvements over established state-of-the-art models, with accuracy gains of up to 3.6\% while conducted statistical analysis using Friedman Aligned-Ranks and Finner post-hoc tests confirms the superiority of our approach across multiple evaluation metrics. |
| title | Forging GEMs: Advancing Greek NLP through Quality-Based Corpus Curation |
| topic | Computation and Language Artificial Intelligence 68T50, 68T07, 68U35 |
| url | https://arxiv.org/abs/2510.20002 |