MSD-Score: Multi-Scale Distributional Scoring for Reference-Free Image Caption Evaluation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913098697277440 |
|---|---|
| author | Kan, Shichao Zhang, Xuyang Zhang, Haojie Zhu, Zhe Cen, Yigang Liang, Yixiong Shan, Lianlei Zhang, Linna Qu, Zhe Xia, Jiazhi |
| author_facet | Kan, Shichao Zhang, Xuyang Zhang, Haojie Zhu, Zhe Cen, Yigang Liang, Yixiong Shan, Lianlei Zhang, Linna Qu, Zhe Xia, Jiazhi |
| contents | Evaluating image captions without references remains challenging because global embedding similarity often misses fine-grained mismatches such as hallucinated objects, missing attributes, or incorrect relations. We propose MSD-Score, a reference-free metric that models image patch and text token embeddings as von Mises-Fisher mixtures on the unit hypersphere. Instead of treating each modality as a single point, MSD-Score formulates image-text matching as a multi-scale distributional scoring problem. Semantic discrepancies are quantified via a weighted bi-directional KL divergence and combined with global similarity in a multi-scale framework for both single- and multi-candidate evaluations. Extensive experiments show that MSD-Score achieves state-of-the-art correlation with human judgments among reference-free metrics. Beyond accuracy, its probabilistic formulation yields transparent and decomposable diagnostics of local grounding errors, providing a deterministic complementary signal to holistic similarity metrics and judge-based evaluators. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_06080 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | MSD-Score: Multi-Scale Distributional Scoring for Reference-Free Image Caption Evaluation Kan, Shichao Zhang, Xuyang Zhang, Haojie Zhu, Zhe Cen, Yigang Liang, Yixiong Shan, Lianlei Zhang, Linna Qu, Zhe Xia, Jiazhi Computer Vision and Pattern Recognition Evaluating image captions without references remains challenging because global embedding similarity often misses fine-grained mismatches such as hallucinated objects, missing attributes, or incorrect relations. We propose MSD-Score, a reference-free metric that models image patch and text token embeddings as von Mises-Fisher mixtures on the unit hypersphere. Instead of treating each modality as a single point, MSD-Score formulates image-text matching as a multi-scale distributional scoring problem. Semantic discrepancies are quantified via a weighted bi-directional KL divergence and combined with global similarity in a multi-scale framework for both single- and multi-candidate evaluations. Extensive experiments show that MSD-Score achieves state-of-the-art correlation with human judgments among reference-free metrics. Beyond accuracy, its probabilistic formulation yields transparent and decomposable diagnostics of local grounding errors, providing a deterministic complementary signal to holistic similarity metrics and judge-based evaluators. |
| title | MSD-Score: Multi-Scale Distributional Scoring for Reference-Free Image Caption Evaluation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2605.06080 |