CaMMT: Benchmarking Culturally Aware Multimodal Machine Translation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911167307317248 |
|---|---|
| author | Villa-Cueva, Emilio Bolatzhanova, Sholpan Turmakhan, Diana Elzeky, Kareem Ademtew, Henok Biadglign Aji, Alham Fikri Araujo, Vladimir Azime, Israel Abebe Baek, Jinheon Belcavello, Frederico Cristobal, Fermin Cruz, Jan Christian Blaise Dabre, Mary Dabre, Raj Ehsan, Toqeer Etori, Naome A Farooqui, Fauzan Geng, Jiahui Ivetta, Guido Jayakumar, Thanmay Jeong, Soyeong Lim, Zheng Wei Mandal, Aishik Martinelli, Sofia Mihaylov, Mihail Minkov Orel, Daniil Pramanick, Aniket Purkayastha, Sukannya Salazar, Israfel Song, Haiyue Torrent, Tiago Timponi Yadeta, Debela Desalegn Hamed, Injy Tonja, Atnafu Lambebo Solorio, Thamar |
| author_facet | Villa-Cueva, Emilio Bolatzhanova, Sholpan Turmakhan, Diana Elzeky, Kareem Ademtew, Henok Biadglign Aji, Alham Fikri Araujo, Vladimir Azime, Israel Abebe Baek, Jinheon Belcavello, Frederico Cristobal, Fermin Cruz, Jan Christian Blaise Dabre, Mary Dabre, Raj Ehsan, Toqeer Etori, Naome A Farooqui, Fauzan Geng, Jiahui Ivetta, Guido Jayakumar, Thanmay Jeong, Soyeong Lim, Zheng Wei Mandal, Aishik Martinelli, Sofia Mihaylov, Mihail Minkov Orel, Daniil Pramanick, Aniket Purkayastha, Sukannya Salazar, Israfel Song, Haiyue Torrent, Tiago Timponi Yadeta, Debela Desalegn Hamed, Injy Tonja, Atnafu Lambebo Solorio, Thamar |
| contents | Translating cultural content poses challenges for machine translation systems due to the differences in conceptualizations between cultures, where language alone may fail to convey sufficient context to capture region-specific meanings. In this work, we investigate whether images can act as cultural context in multimodal translation. We introduce CaMMT, a human-curated benchmark of over 5,800 triples of images along with parallel captions in English and regional languages. Using this dataset, we evaluate five Vision Language Models (VLMs) in text-only and text+image settings. Through automatic and human evaluations, we find that visual context generally improves translation quality, especially in handling Culturally-Specific Items (CSIs), disambiguation, and correct gender marking. By releasing CaMMT, our objective is to support broader efforts to build and evaluate multimodal translation systems that are better aligned with cultural nuance and regional variations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_24456 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | CaMMT: Benchmarking Culturally Aware Multimodal Machine Translation Villa-Cueva, Emilio Bolatzhanova, Sholpan Turmakhan, Diana Elzeky, Kareem Ademtew, Henok Biadglign Aji, Alham Fikri Araujo, Vladimir Azime, Israel Abebe Baek, Jinheon Belcavello, Frederico Cristobal, Fermin Cruz, Jan Christian Blaise Dabre, Mary Dabre, Raj Ehsan, Toqeer Etori, Naome A Farooqui, Fauzan Geng, Jiahui Ivetta, Guido Jayakumar, Thanmay Jeong, Soyeong Lim, Zheng Wei Mandal, Aishik Martinelli, Sofia Mihaylov, Mihail Minkov Orel, Daniil Pramanick, Aniket Purkayastha, Sukannya Salazar, Israfel Song, Haiyue Torrent, Tiago Timponi Yadeta, Debela Desalegn Hamed, Injy Tonja, Atnafu Lambebo Solorio, Thamar Computation and Language Translating cultural content poses challenges for machine translation systems due to the differences in conceptualizations between cultures, where language alone may fail to convey sufficient context to capture region-specific meanings. In this work, we investigate whether images can act as cultural context in multimodal translation. We introduce CaMMT, a human-curated benchmark of over 5,800 triples of images along with parallel captions in English and regional languages. Using this dataset, we evaluate five Vision Language Models (VLMs) in text-only and text+image settings. Through automatic and human evaluations, we find that visual context generally improves translation quality, especially in handling Culturally-Specific Items (CSIs), disambiguation, and correct gender marking. By releasing CaMMT, our objective is to support broader efforts to build and evaluate multimodal translation systems that are better aligned with cultural nuance and regional variations. |
| title | CaMMT: Benchmarking Culturally Aware Multimodal Machine Translation |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2505.24456 |