CreoleVal: Multilingual Multitask Benchmarks for Creoles
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2023
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913342152507392 |
|---|---|
| author | Lent, Heather Tatariya, Kushal Dabre, Raj Chen, Yiyi Fekete, Marcell Ploeger, Esther Zhou, Li Armstrong, Ruth-Ann Eijansantos, Abee Malau, Catriona Heje, Hans Erik Lavrinovics, Ernests Kanojia, Diptesh Belony, Paul Bollmann, Marcel Grobol, Loïc de Lhoneux, Miryam Hershcovich, Daniel DeGraff, Michel Søgaard, Anders Bjerva, Johannes |
| author_facet | Lent, Heather Tatariya, Kushal Dabre, Raj Chen, Yiyi Fekete, Marcell Ploeger, Esther Zhou, Li Armstrong, Ruth-Ann Eijansantos, Abee Malau, Catriona Heje, Hans Erik Lavrinovics, Ernests Kanojia, Diptesh Belony, Paul Bollmann, Marcel Grobol, Loïc de Lhoneux, Miryam Hershcovich, Daniel DeGraff, Michel Søgaard, Anders Bjerva, Johannes |
| contents | Creoles represent an under-explored and marginalized group of languages, with few available resources for NLP research.While the genealogical ties between Creoles and a number of highly-resourced languages imply a significant potential for transfer learning, this potential is hampered due to this lack of annotated data. In this work we present CreoleVal, a collection of benchmark datasets spanning 8 different NLP tasks, covering up to 28 Creole languages; it is an aggregate of novel development datasets for reading comprehension, relation classification, and machine translation for Creoles, in addition to a practical gateway to a handful of preexisting benchmarks. For each benchmark, we conduct baseline experiments in a zero-shot setting in order to further ascertain the capabilities and limitations of transfer learning for Creoles. Ultimately, we see CreoleVal as an opportunity to empower research on Creoles in NLP and computational linguistics, and in general, a step towards more equitable language technology around the globe. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_19567 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | CreoleVal: Multilingual Multitask Benchmarks for Creoles Lent, Heather Tatariya, Kushal Dabre, Raj Chen, Yiyi Fekete, Marcell Ploeger, Esther Zhou, Li Armstrong, Ruth-Ann Eijansantos, Abee Malau, Catriona Heje, Hans Erik Lavrinovics, Ernests Kanojia, Diptesh Belony, Paul Bollmann, Marcel Grobol, Loïc de Lhoneux, Miryam Hershcovich, Daniel DeGraff, Michel Søgaard, Anders Bjerva, Johannes Computation and Language Artificial Intelligence Creoles represent an under-explored and marginalized group of languages, with few available resources for NLP research.While the genealogical ties between Creoles and a number of highly-resourced languages imply a significant potential for transfer learning, this potential is hampered due to this lack of annotated data. In this work we present CreoleVal, a collection of benchmark datasets spanning 8 different NLP tasks, covering up to 28 Creole languages; it is an aggregate of novel development datasets for reading comprehension, relation classification, and machine translation for Creoles, in addition to a practical gateway to a handful of preexisting benchmarks. For each benchmark, we conduct baseline experiments in a zero-shot setting in order to further ascertain the capabilities and limitations of transfer learning for Creoles. Ultimately, we see CreoleVal as an opportunity to empower research on Creoles in NLP and computational linguistics, and in general, a step towards more equitable language technology around the globe. |
| title | CreoleVal: Multilingual Multitask Benchmarks for Creoles |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2310.19567 |