Donors and Recipients: On Asymmetric Transfer Across Tasks and Languages with Parameter-Efficient Fine-Tuning
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866918278120603648 |
|---|---|
| author | Dymkiewicz, Kajetan Vulic, Ivan Yannakoudakis, Helen Shapira, Eilam Reichart, Roi Korhonen, Anna |
| author_facet | Dymkiewicz, Kajetan Vulic, Ivan Yannakoudakis, Helen Shapira, Eilam Reichart, Roi Korhonen, Anna |
| contents | Large language models (LLMs) perform strongly across tasks and languages, yet how improvements in one task or language affect other tasks and languages remains poorly understood. We conduct a controlled LoRA fine-tuning study across multiple open-weight LLM families and scales, using a standardised grid of 11 languages and four benchmarks. We fine-tune each model on a single task-language source and measure transfer when evaluated on all other task-language target pairs. We decompose transfer into three regimes: (i) Matched-Task (Cross-Language), (ii) Matched-Language (Cross-Task), and (iii) Cross-Task (Cross-Language). Single-source fine-tuning yields a net positive uplift across regimes, but the gains are strongly asymmetric. Matched-Task (Cross-Language) transfer emerges as the most effective and predictable regime, driven principally by the identity of the target language rather than model architecture. We identify a stable hierarchy where high-resource languages and broad semantic tasks act as efficient recipients that absorb gains from diverse sources, while specialised tasks and lower-resource languages are more isolated. These results imply that effective fine-tuning requires navigating donor-recipient roles to maximise downstream gains. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_13368 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Donors and Recipients: On Asymmetric Transfer Across Tasks and Languages with Parameter-Efficient Fine-Tuning Dymkiewicz, Kajetan Vulic, Ivan Yannakoudakis, Helen Shapira, Eilam Reichart, Roi Korhonen, Anna Computation and Language Artificial Intelligence Large language models (LLMs) perform strongly across tasks and languages, yet how improvements in one task or language affect other tasks and languages remains poorly understood. We conduct a controlled LoRA fine-tuning study across multiple open-weight LLM families and scales, using a standardised grid of 11 languages and four benchmarks. We fine-tune each model on a single task-language source and measure transfer when evaluated on all other task-language target pairs. We decompose transfer into three regimes: (i) Matched-Task (Cross-Language), (ii) Matched-Language (Cross-Task), and (iii) Cross-Task (Cross-Language). Single-source fine-tuning yields a net positive uplift across regimes, but the gains are strongly asymmetric. Matched-Task (Cross-Language) transfer emerges as the most effective and predictable regime, driven principally by the identity of the target language rather than model architecture. We identify a stable hierarchy where high-resource languages and broad semantic tasks act as efficient recipients that absorb gains from diverse sources, while specialised tasks and lower-resource languages are more isolated. These results imply that effective fine-tuning requires navigating donor-recipient roles to maximise downstream gains. |
| title | Donors and Recipients: On Asymmetric Transfer Across Tasks and Languages with Parameter-Efficient Fine-Tuning |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2511.13368 |