SemEval-2025 Task 1: AdMIRe -- Advancing Multimodal Idiomaticity Representation
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866908392184872960 |
|---|---|
| author | Pickard, Thomas Villavicencio, Aline Mi, Maggie He, Wei Phelps, Dylan Idiart, Marco |
| author_facet | Pickard, Thomas Villavicencio, Aline Mi, Maggie He, Wei Phelps, Dylan Idiart, Marco |
| contents | Idiomatic expressions present a unique challenge in NLP, as their meanings are often not directly inferable from their constituent words. Despite recent advancements in Large Language Models (LLMs), idiomaticity remains a significant obstacle to robust semantic representation. We present datasets and tasks for SemEval-2025 Task 1: AdMiRe (Advancing Multimodal Idiomaticity Representation), which challenges the community to assess and improve models' ability to interpret idiomatic expressions in multimodal contexts and in multiple languages. Participants competed in two subtasks: ranking images based on their alignment with idiomatic or literal meanings, and predicting the next image in a sequence. The most effective methods achieved human-level performance by leveraging pretrained LLMs and vision-language models in mixture-of-experts settings, with multiple queries used to smooth over the weaknesses in these models' representations of idiomaticity. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_15358 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SemEval-2025 Task 1: AdMIRe -- Advancing Multimodal Idiomaticity Representation Pickard, Thomas Villavicencio, Aline Mi, Maggie He, Wei Phelps, Dylan Idiart, Marco Computation and Language Computer Vision and Pattern Recognition I.2.7; I.4.m Idiomatic expressions present a unique challenge in NLP, as their meanings are often not directly inferable from their constituent words. Despite recent advancements in Large Language Models (LLMs), idiomaticity remains a significant obstacle to robust semantic representation. We present datasets and tasks for SemEval-2025 Task 1: AdMiRe (Advancing Multimodal Idiomaticity Representation), which challenges the community to assess and improve models' ability to interpret idiomatic expressions in multimodal contexts and in multiple languages. Participants competed in two subtasks: ranking images based on their alignment with idiomatic or literal meanings, and predicting the next image in a sequence. The most effective methods achieved human-level performance by leveraging pretrained LLMs and vision-language models in mixture-of-experts settings, with multiple queries used to smooth over the weaknesses in these models' representations of idiomaticity. |
| title | SemEval-2025 Task 1: AdMIRe -- Advancing Multimodal Idiomaticity Representation |
| topic | Computation and Language Computer Vision and Pattern Recognition I.2.7; I.4.m |
| url | https://arxiv.org/abs/2503.15358 |