Multi-label Scandinavian Language Identification (SLIDE)
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910820359733248 |
|---|---|
| author | Fedorova, Mariia Frydenberg, Jonas Sebulon Handford, Victoria Langø, Victoria Ovedie Chruickshank Willoch, Solveig Helene Midtgaard, Marthe Løken Scherrer, Yves Mæhlum, Petter Samuel, David |
| author_facet | Fedorova, Mariia Frydenberg, Jonas Sebulon Handford, Victoria Langø, Victoria Ovedie Chruickshank Willoch, Solveig Helene Midtgaard, Marthe Løken Scherrer, Yves Mæhlum, Petter Samuel, David |
| contents | Identifying closely related languages at sentence level is difficult, in particular because it is often impossible to assign a sentence to a single language. In this paper, we focus on multi-label sentence-level Scandinavian language identification (LID) for Danish, Norwegian Bokmål, Norwegian Nynorsk, and Swedish. We present the Scandinavian Language Identification and Evaluation, SLIDE, a manually curated multi-label evaluation dataset and a suite of LID models with varying speed-accuracy tradeoffs. We demonstrate that the ability to identify multiple languages simultaneously is necessary for any accurate LID method, and present a novel approach to training such multi-label LID models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_06692 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Multi-label Scandinavian Language Identification (SLIDE) Fedorova, Mariia Frydenberg, Jonas Sebulon Handford, Victoria Langø, Victoria Ovedie Chruickshank Willoch, Solveig Helene Midtgaard, Marthe Løken Scherrer, Yves Mæhlum, Petter Samuel, David Computation and Language Artificial Intelligence Identifying closely related languages at sentence level is difficult, in particular because it is often impossible to assign a sentence to a single language. In this paper, we focus on multi-label sentence-level Scandinavian language identification (LID) for Danish, Norwegian Bokmål, Norwegian Nynorsk, and Swedish. We present the Scandinavian Language Identification and Evaluation, SLIDE, a manually curated multi-label evaluation dataset and a suite of LID models with varying speed-accuracy tradeoffs. We demonstrate that the ability to identify multiple languages simultaneously is necessary for any accurate LID method, and present a novel approach to training such multi-label LID models. |
| title | Multi-label Scandinavian Language Identification (SLIDE) |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2502.06692 |