Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2404.02452 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916191070584832 |
|---|---|
| author | Villa-Cueva, Emilio López-Monroy, A. Pastor Sánchez-Vega, Fernando Solorio, Thamar |
| author_facet | Villa-Cueva, Emilio López-Monroy, A. Pastor Sánchez-Vega, Fernando Solorio, Thamar |
| contents | Zero-Shot Cross-lingual Transfer (ZS-XLT) utilizes a model trained in a source language to make predictions in another language, often with a performance loss. To alleviate this, additional improvements can be achieved through subsequent adaptation using examples in the target language. In this paper, we exploit In-Context Tuning (ICT) for One-Shot Cross-lingual transfer in the classification task by introducing In-Context Cross-lingual Transfer (IC-XLT). The novel concept involves training a model to learn from context examples and subsequently adapting it during inference to a target language by prepending a One-Shot context demonstration in that language. Our results show that IC-XLT successfully leverages target-language examples to improve the cross-lingual capabilities of the evaluated mT5 model, outperforming prompt-based models in the Zero and Few-shot scenarios adapted through fine-tuning. Moreover, we show that when source-language data is limited, the fine-tuning framework employed for IC-XLT performs comparably to prompt-based fine-tuning with significantly more training data in the source language. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_02452 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Adaptive Cross-lingual Text Classification through In-Context One-Shot Demonstrations Villa-Cueva, Emilio López-Monroy, A. Pastor Sánchez-Vega, Fernando Solorio, Thamar Computation and Language Zero-Shot Cross-lingual Transfer (ZS-XLT) utilizes a model trained in a source language to make predictions in another language, often with a performance loss. To alleviate this, additional improvements can be achieved through subsequent adaptation using examples in the target language. In this paper, we exploit In-Context Tuning (ICT) for One-Shot Cross-lingual transfer in the classification task by introducing In-Context Cross-lingual Transfer (IC-XLT). The novel concept involves training a model to learn from context examples and subsequently adapting it during inference to a target language by prepending a One-Shot context demonstration in that language. Our results show that IC-XLT successfully leverages target-language examples to improve the cross-lingual capabilities of the evaluated mT5 model, outperforming prompt-based models in the Zero and Few-shot scenarios adapted through fine-tuning. Moreover, we show that when source-language data is limited, the fine-tuning framework employed for IC-XLT performs comparably to prompt-based fine-tuning with significantly more training data in the source language. |
| title | Adaptive Cross-lingual Text Classification through In-Context One-Shot Demonstrations |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2404.02452 |