Interference Matrix: Quantifying Cross-Lingual Interference in Transformer Encoders
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915425406681088 |
|---|---|
| author | Alastruey, Belen Janeiro, João Maria Allauzen, Alexandre Elbayad, Maha Barrault, Loïc Costa-jussà, Marta R. |
| author_facet | Alastruey, Belen Janeiro, João Maria Allauzen, Alexandre Elbayad, Maha Barrault, Loïc Costa-jussà, Marta R. |
| contents | In this paper, we present a comprehensive study of language interference in encoder-only Transformer models across 83 languages. We construct an interference matrix by training and evaluating small BERT-like models on all possible language pairs, providing a large-scale quantification of cross-lingual interference. Our analysis reveals that interference between languages is asymmetrical and that its patterns do not align with traditional linguistic characteristics, such as language family, nor with proxies like embedding similarity, but instead better relate to script. Finally, we demonstrate that the interference matrix effectively predicts performance on downstream tasks, serving as a tool to better design multilingual models to obtain optimal performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_02256 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Interference Matrix: Quantifying Cross-Lingual Interference in Transformer Encoders Alastruey, Belen Janeiro, João Maria Allauzen, Alexandre Elbayad, Maha Barrault, Loïc Costa-jussà, Marta R. Computation and Language In this paper, we present a comprehensive study of language interference in encoder-only Transformer models across 83 languages. We construct an interference matrix by training and evaluating small BERT-like models on all possible language pairs, providing a large-scale quantification of cross-lingual interference. Our analysis reveals that interference between languages is asymmetrical and that its patterns do not align with traditional linguistic characteristics, such as language family, nor with proxies like embedding similarity, but instead better relate to script. Finally, we demonstrate that the interference matrix effectively predicts performance on downstream tasks, serving as a tool to better design multilingual models to obtain optimal performance. |
| title | Interference Matrix: Quantifying Cross-Lingual Interference in Transformer Encoders |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2508.02256 |