Cross-Lingual Representation Alignment Through Contrastive Image-Caption Tuning
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913847919509504 |
|---|---|
| author | Krasner, Nathaniel Lanuzo, Nicholas Anastasopoulos, Antonios |
| author_facet | Krasner, Nathaniel Lanuzo, Nicholas Anastasopoulos, Antonios |
| contents | Multilingual alignment of sentence representations has mostly required bitexts to bridge the gap between languages. We investigate whether visual information can bridge this gap instead. Image caption datasets are very easy to create without requiring multilingual expertise, so this offers a more efficient alternative for low-resource languages. We find that multilingual image-caption alignment can implicitly align the text representations between languages, languages unseen by the encoder in pretraining can be incorporated into this alignment post-hoc, and these aligned representations are usable for cross-lingual Natural Language Understanding (NLU) and bitext retrieval. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_13628 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Cross-Lingual Representation Alignment Through Contrastive Image-Caption Tuning Krasner, Nathaniel Lanuzo, Nicholas Anastasopoulos, Antonios Computation and Language Multilingual alignment of sentence representations has mostly required bitexts to bridge the gap between languages. We investigate whether visual information can bridge this gap instead. Image caption datasets are very easy to create without requiring multilingual expertise, so this offers a more efficient alternative for low-resource languages. We find that multilingual image-caption alignment can implicitly align the text representations between languages, languages unseen by the encoder in pretraining can be incorporated into this alignment post-hoc, and these aligned representations are usable for cross-lingual Natural Language Understanding (NLU) and bitext retrieval. |
| title | Cross-Lingual Representation Alignment Through Contrastive Image-Caption Tuning |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2505.13628 |