BMIKE-53: Investigating Cross-Lingual Knowledge Editing with In-Context Learning
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912404429864960 |
|---|---|
| author | Nie, Ercong Shao, Bo Ding, Zifeng Wang, Mingyang Schmid, Helmut Schütze, Hinrich |
| author_facet | Nie, Ercong Shao, Bo Ding, Zifeng Wang, Mingyang Schmid, Helmut Schütze, Hinrich |
| contents | This paper introduces BMIKE-53, a comprehensive benchmark for cross-lingual in-context knowledge editing (IKE) across 53 languages, unifying three knowledge editing (KE) datasets: zsRE, CounterFact, and WikiFactDiff. Cross-lingual KE, which requires knowledge edited in one language to generalize across others while preserving unrelated knowledge, remains underexplored. To address this gap, we systematically evaluate IKE under zero-shot, one-shot, and few-shot setups, incorporating tailored metric-specific demonstrations. Our findings reveal that model scale and demonstration alignment critically govern cross-lingual IKE efficacy, with larger models and tailored demonstrations significantly improving performance. Linguistic properties, particularly script type, strongly influence performance variation across languages, with non-Latin languages underperforming due to issues like language confusion. Code and data are publicly available at: https://github.com/ercong21/MultiKnow/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_17764 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | BMIKE-53: Investigating Cross-Lingual Knowledge Editing with In-Context Learning Nie, Ercong Shao, Bo Ding, Zifeng Wang, Mingyang Schmid, Helmut Schütze, Hinrich Computation and Language Artificial Intelligence This paper introduces BMIKE-53, a comprehensive benchmark for cross-lingual in-context knowledge editing (IKE) across 53 languages, unifying three knowledge editing (KE) datasets: zsRE, CounterFact, and WikiFactDiff. Cross-lingual KE, which requires knowledge edited in one language to generalize across others while preserving unrelated knowledge, remains underexplored. To address this gap, we systematically evaluate IKE under zero-shot, one-shot, and few-shot setups, incorporating tailored metric-specific demonstrations. Our findings reveal that model scale and demonstration alignment critically govern cross-lingual IKE efficacy, with larger models and tailored demonstrations significantly improving performance. Linguistic properties, particularly script type, strongly influence performance variation across languages, with non-Latin languages underperforming due to issues like language confusion. Code and data are publicly available at: https://github.com/ercong21/MultiKnow/. |
| title | BMIKE-53: Investigating Cross-Lingual Knowledge Editing with In-Context Learning |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2406.17764 |