Through the Looking Glass: Common Sense Consistency Evaluation of Weird Images
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910938792198144 |
|---|---|
| author | Rykov, Elisei Petrushina, Kseniia Titova, Kseniia Razzhigaev, Anton Panchenko, Alexander Konovalov, Vasily |
| author_facet | Rykov, Elisei Petrushina, Kseniia Titova, Kseniia Razzhigaev, Anton Panchenko, Alexander Konovalov, Vasily |
| contents | Measuring how real images look is a complex task in artificial intelligence research. For example, an image of a boy with a vacuum cleaner in a desert violates common sense. We introduce a novel method, which we call Through the Looking Glass (TLG), to assess image common sense consistency using Large Vision-Language Models (LVLMs) and Transformer-based encoder. By leveraging LVLMs to extract atomic facts from these images, we obtain a mix of accurate facts. We proceed by fine-tuning a compact attention-pooling classifier over encoded atomic facts. Our TLG has achieved a new state-of-the-art performance on the WHOOPS! and WEIRD datasets while leveraging a compact fine-tuning component. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_07704 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Through the Looking Glass: Common Sense Consistency Evaluation of Weird Images Rykov, Elisei Petrushina, Kseniia Titova, Kseniia Razzhigaev, Anton Panchenko, Alexander Konovalov, Vasily Computer Vision and Pattern Recognition Computation and Language Measuring how real images look is a complex task in artificial intelligence research. For example, an image of a boy with a vacuum cleaner in a desert violates common sense. We introduce a novel method, which we call Through the Looking Glass (TLG), to assess image common sense consistency using Large Vision-Language Models (LVLMs) and Transformer-based encoder. By leveraging LVLMs to extract atomic facts from these images, we obtain a mix of accurate facts. We proceed by fine-tuning a compact attention-pooling classifier over encoded atomic facts. Our TLG has achieved a new state-of-the-art performance on the WHOOPS! and WEIRD datasets while leveraging a compact fine-tuning component. |
| title | Through the Looking Glass: Common Sense Consistency Evaluation of Weird Images |
| topic | Computer Vision and Pattern Recognition Computation and Language |
| url | https://arxiv.org/abs/2505.07704 |