DISCODE: Distribution-Aware Score Decoder for Robust Automatic Evaluation of Image Captioning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915706372620288 |
|---|---|
| author | Inoue, Nakamasa Goto, Kanoko Oi, Masanari Gruszka, Martyna Ukai, Mahiro Hirose, Takumi Sekikawa, Yusuke |
| author_facet | Inoue, Nakamasa Goto, Kanoko Oi, Masanari Gruszka, Martyna Ukai, Mahiro Hirose, Takumi Sekikawa, Yusuke |
| contents | Large vision-language models (LVLMs) have shown impressive performance across a broad range of multimodal tasks. However, robust image caption evaluation using LVLMs remains challenging, particularly under domain-shift scenarios. To address this issue, we introduce the Distribution-Aware Score Decoder (DISCODE), a novel finetuning-free method that generates robust evaluation scores better aligned with human judgments across diverse domains. The core idea behind DISCODE lies in its test-time adaptive evaluation approach, which introduces the Adaptive Test-Time (ATT) loss, leveraging a Gaussian prior distribution to improve robustness in evaluation score estimation. This loss is efficiently minimized at test time using an analytical solution that we derive. Furthermore, we introduce the Multi-domain Caption Evaluation (MCEval) benchmark, a new image captioning evaluation benchmark covering six distinct domains, designed to assess the robustness of evaluation metrics. In our experiments, we demonstrate that DISCODE achieves state-of-the-art performance as a reference-free evaluation metric across MCEval and four representative existing benchmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_14420 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DISCODE: Distribution-Aware Score Decoder for Robust Automatic Evaluation of Image Captioning Inoue, Nakamasa Goto, Kanoko Oi, Masanari Gruszka, Martyna Ukai, Mahiro Hirose, Takumi Sekikawa, Yusuke Computer Vision and Pattern Recognition Artificial Intelligence Large vision-language models (LVLMs) have shown impressive performance across a broad range of multimodal tasks. However, robust image caption evaluation using LVLMs remains challenging, particularly under domain-shift scenarios. To address this issue, we introduce the Distribution-Aware Score Decoder (DISCODE), a novel finetuning-free method that generates robust evaluation scores better aligned with human judgments across diverse domains. The core idea behind DISCODE lies in its test-time adaptive evaluation approach, which introduces the Adaptive Test-Time (ATT) loss, leveraging a Gaussian prior distribution to improve robustness in evaluation score estimation. This loss is efficiently minimized at test time using an analytical solution that we derive. Furthermore, we introduce the Multi-domain Caption Evaluation (MCEval) benchmark, a new image captioning evaluation benchmark covering six distinct domains, designed to assess the robustness of evaluation metrics. In our experiments, we demonstrate that DISCODE achieves state-of-the-art performance as a reference-free evaluation metric across MCEval and four representative existing benchmarks. |
| title | DISCODE: Distribution-Aware Score Decoder for Robust Automatic Evaluation of Image Captioning |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2512.14420 |