Exploring Language-Agnosticity in Function Vectors: A Case Study in Machine Translation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917426553159680 |
|---|---|
| author | Laiyk, Nurkhan Gállego, Gerard I. Ferrando, Javier Koto, Fajri |
| author_facet | Laiyk, Nurkhan Gállego, Gerard I. Ferrando, Javier Koto, Fajri |
| contents | Function vectors (FVs) are vector representations of tasks extracted from model activations during in-context learning. While prior work has shown that multilingual model representations can be language-agnostic, it remains unclear whether the same holds for function vectors. We study whether FVs exhibit language-agnosticity, using machine translation as a case study. Across three decoder-only multilingual LLMs, we find that translation FVs extracted from a single English$\rightarrow$Target direction transfer to other target languages, consistently improving the rank of correct translation tokens across multiple unseen languages. Ablation results show that removing the FV degrades translation across languages with limited impact on unrelated tasks. We further show that base-model FVs transfer to instruction-tuned variants and partially generalize from word-level to sentence-level translation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_19678 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Exploring Language-Agnosticity in Function Vectors: A Case Study in Machine Translation Laiyk, Nurkhan Gállego, Gerard I. Ferrando, Javier Koto, Fajri Computation and Language Function vectors (FVs) are vector representations of tasks extracted from model activations during in-context learning. While prior work has shown that multilingual model representations can be language-agnostic, it remains unclear whether the same holds for function vectors. We study whether FVs exhibit language-agnosticity, using machine translation as a case study. Across three decoder-only multilingual LLMs, we find that translation FVs extracted from a single English$\rightarrow$Target direction transfer to other target languages, consistently improving the rank of correct translation tokens across multiple unseen languages. Ablation results show that removing the FV degrades translation across languages with limited impact on unrelated tasks. We further show that base-model FVs transfer to instruction-tuned variants and partially generalize from word-level to sentence-level translation. |
| title | Exploring Language-Agnosticity in Function Vectors: A Case Study in Machine Translation |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2604.19678 |