Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2409.17373 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916411579826176 |
|---|---|
| author | Amirzadeh, Hamidreza Jafari, Sadegh Harju, Anika van der Goot, Rob |
| author_facet | Amirzadeh, Hamidreza Jafari, Sadegh Harju, Anika van der Goot, Rob |
| contents | Language typology databases enhance multi-lingual Natural Language Processing (NLP) by improving model adaptability to diverse linguistic structures. The widely-used lang2vec toolkit integrates several such databases, but its coverage remains limited at 28.9\%. Previous work on automatically increasing coverage predicts missing values based on features from other languages or focuses on single features, we propose to use textual data for better-informed feature prediction. To this end, we introduce a multi-lingual Part-of-Speech (POS) tagger, achieving over 70\% accuracy across 1,749 languages, and experiment with external statistical features and a variety of machine learning algorithms. We also introduce a more realistic evaluation setup, focusing on likely to be missing typology features, and show that our approach outperforms previous work in both setups. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_17373 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | data2lang2vec: Data Driven Typological Features Completion Amirzadeh, Hamidreza Jafari, Sadegh Harju, Anika van der Goot, Rob Computation and Language Language typology databases enhance multi-lingual Natural Language Processing (NLP) by improving model adaptability to diverse linguistic structures. The widely-used lang2vec toolkit integrates several such databases, but its coverage remains limited at 28.9\%. Previous work on automatically increasing coverage predicts missing values based on features from other languages or focuses on single features, we propose to use textual data for better-informed feature prediction. To this end, we introduce a multi-lingual Part-of-Speech (POS) tagger, achieving over 70\% accuracy across 1,749 languages, and experiment with external statistical features and a variety of machine learning algorithms. We also introduce a more realistic evaluation setup, focusing on likely to be missing typology features, and show that our approach outperforms previous work in both setups. |
| title | data2lang2vec: Data Driven Typological Features Completion |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2409.17373 |