KréyoLID From Language Identification Towards Language Mining
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910865409703936 |
|---|---|
| author | Dent, Rasul Suarez, Pedro Ortiz Clérice, Thibault Sagot, Benoît |
| author_facet | Dent, Rasul Suarez, Pedro Ortiz Clérice, Thibault Sagot, Benoît |
| contents | Automatic language identification is frequently framed as a multi-class classification problem. However, when creating digital corpora for less commonly written languages, it may be more appropriate to consider it a data mining problem. For these varieties, one knows ahead of time that the vast majority of documents are of little interest. By minimizing resources spent on classifying such documents, we can create corpora much faster and with better coverage than using established pipelines. To demonstrate the effectiveness of the language mining perspective, we introduce a new pipeline and corpora for several French-based Creoles. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_06547 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | KréyoLID From Language Identification Towards Language Mining Dent, Rasul Suarez, Pedro Ortiz Clérice, Thibault Sagot, Benoît Computation and Language Automatic language identification is frequently framed as a multi-class classification problem. However, when creating digital corpora for less commonly written languages, it may be more appropriate to consider it a data mining problem. For these varieties, one knows ahead of time that the vast majority of documents are of little interest. By minimizing resources spent on classifying such documents, we can create corpora much faster and with better coverage than using established pipelines. To demonstrate the effectiveness of the language mining perspective, we introduce a new pipeline and corpora for several French-based Creoles. |
| title | KréyoLID From Language Identification Towards Language Mining |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2503.06547 |