Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2603.23627 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911543076061184 |
|---|---|
| author | Laba, Yurii Mohytych, Yaryna Rohulia, Ivanna Kyryleyza, Halyna Dydyk-Meush, Hanna Dobosevych, Oles Hryniv, Rostyslav |
| author_facet | Laba, Yurii Mohytych, Yaryna Rohulia, Ivanna Kyryleyza, Halyna Dydyk-Meush, Hanna Dobosevych, Oles Hryniv, Rostyslav |
| contents | This study presents a benchmark for evaluating the Visual Word Sense Disambiguation (Visual-WSD) task in Ukrainian. The main goal of the Visual-WSD task is to identify, with minimal contextual information, the most appropriate representation of a given ambiguous word from a set of ten images. To construct this benchmark, we followed a methodology similar to that proposed by (CITATION), who previously introduced benchmarks for the Visual-WSD task in English, Italian, and Farsi. This approach allows us to incorporate the Ukrainian benchmark into a broader framework for cross-language model performance comparisons. We collected the benchmark data semi-automatically and refined it with input from domain experts. We then assessed eight multilingual and multimodal large language models using this benchmark. All tested models performed worse than the zero-shot CLIP-based baseline model (CITATION) used by (CITATION) for the English Visual-WSD task. Our analysis revealed a significant performance gap in the Visual-WSD task between Ukrainian and English. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_23627 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Ukrainian Visual Word Sense Disambiguation Benchmark Laba, Yurii Mohytych, Yaryna Rohulia, Ivanna Kyryleyza, Halyna Dydyk-Meush, Hanna Dobosevych, Oles Hryniv, Rostyslav Computer Vision and Pattern Recognition Artificial Intelligence This study presents a benchmark for evaluating the Visual Word Sense Disambiguation (Visual-WSD) task in Ukrainian. The main goal of the Visual-WSD task is to identify, with minimal contextual information, the most appropriate representation of a given ambiguous word from a set of ten images. To construct this benchmark, we followed a methodology similar to that proposed by (CITATION), who previously introduced benchmarks for the Visual-WSD task in English, Italian, and Farsi. This approach allows us to incorporate the Ukrainian benchmark into a broader framework for cross-language model performance comparisons. We collected the benchmark data semi-automatically and refined it with input from domain experts. We then assessed eight multilingual and multimodal large language models using this benchmark. All tested models performed worse than the zero-shot CLIP-based baseline model (CITATION) used by (CITATION) for the English Visual-WSD task. Our analysis revealed a significant performance gap in the Visual-WSD task between Ukrainian and English. |
| title | Ukrainian Visual Word Sense Disambiguation Benchmark |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2603.23627 |