Can Argus Judge Them All? Comparing VLMs Across Domains
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915368315912192 |
|---|---|
| author | Joshi, Harsh Kashyap, Gautam Siddharth Ali, Rafiq Shabbir, Ebad Jain, Niharika Jain, Sarthak Gao, Jiechao Naseem, Usman |
| author_facet | Joshi, Harsh Kashyap, Gautam Siddharth Ali, Rafiq Shabbir, Ebad Jain, Niharika Jain, Sarthak Gao, Jiechao Naseem, Usman |
| contents | Vision-Language Models (VLMs) are advancing multimodal AI, yet their performance consistency across tasks is underexamined. We benchmark CLIP, BLIP, and LXMERT across diverse datasets spanning retrieval, captioning, and reasoning. Our evaluation includes task accuracy, generation quality, efficiency, and a novel Cross-Dataset Consistency (CDC) metric. CLIP shows strongest generalization (CDC: 0.92), BLIP excels on curated data, and LXMERT leads in structured reasoning. These results expose trade-offs between generalization and specialization, informing industrial deployment of VLMs and guiding development toward robust, task-flexible architectures. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_01042 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Can Argus Judge Them All? Comparing VLMs Across Domains Joshi, Harsh Kashyap, Gautam Siddharth Ali, Rafiq Shabbir, Ebad Jain, Niharika Jain, Sarthak Gao, Jiechao Naseem, Usman Information Retrieval Artificial Intelligence Computation and Language Vision-Language Models (VLMs) are advancing multimodal AI, yet their performance consistency across tasks is underexamined. We benchmark CLIP, BLIP, and LXMERT across diverse datasets spanning retrieval, captioning, and reasoning. Our evaluation includes task accuracy, generation quality, efficiency, and a novel Cross-Dataset Consistency (CDC) metric. CLIP shows strongest generalization (CDC: 0.92), BLIP excels on curated data, and LXMERT leads in structured reasoning. These results expose trade-offs between generalization and specialization, informing industrial deployment of VLMs and guiding development toward robust, task-flexible architectures. |
| title | Can Argus Judge Them All? Comparing VLMs Across Domains |
| topic | Information Retrieval Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2507.01042 |