Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2606.02535 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917556444463104 |
|---|---|
| author | Liu, Lu Duan, Huiyu Zhu, Chenxin Lu, Jintong Jiang, Haoyun Yang, Liu Hu, Qiang Zhai, Guangtao Zhang, Xiaoyun |
| author_facet | Liu, Lu Duan, Huiyu Zhu, Chenxin Lu, Jintong Jiang, Haoyun Yang, Liu Hu, Qiang Zhai, Guangtao Zhang, Xiaoyun |
| contents | Large-scale generative models have demonstrated remarkable capabilities across image generation and editing tasks. However, their performance in low-level vision tasks, which require pixel-wise control, remains insufficiently studied. To address this gap, we introduce \textbf{LL-Bench}, a comprehensive \textbf{Benchmark} for evaluating the capabilities of large-scale generative models on \textbf{L}ow-\textbf{L}evel vision tasks. The benchmark comprises 2,469 real-world degraded images covering 16 low-level degradation tasks, and 28,919 restored images produced by 10 state-of-the-art large-scale generative models and 21 conventional restoration models, which are annotated with 152,020 expert-level pairwise human preferences and 28,334 quality scores. Built upon LL-Bench, we present a systematic diagnosis that reveals the performance boundaries and unique failure modes of large-scale generative models across diverse low-level vision tasks, compared with conventional representative restoration approaches. Moreover, we investigate the effectiveness of current quality evaluation metrics on LL-Bench, which exhibit significant discrepancy with human ratings. To better align restored-image quality assessment with human preferences, we further propose \textbf{LL-Score}, an MLLM-based evaluator that captures both restoration quality and hallucination existence. Extensive experiments demonstrate that LL-score not only outperforms existing image quality assessment metrics, but also serves as a promising reward model for training generative models on low-level vision tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2606_02535 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | LL-Bench: Rethinking Low-Level Vision Evaluation in the Era of Large-Scale Generative Models Liu, Lu Duan, Huiyu Zhu, Chenxin Lu, Jintong Jiang, Haoyun Yang, Liu Hu, Qiang Zhai, Guangtao Zhang, Xiaoyun Computer Vision and Pattern Recognition Large-scale generative models have demonstrated remarkable capabilities across image generation and editing tasks. However, their performance in low-level vision tasks, which require pixel-wise control, remains insufficiently studied. To address this gap, we introduce \textbf{LL-Bench}, a comprehensive \textbf{Benchmark} for evaluating the capabilities of large-scale generative models on \textbf{L}ow-\textbf{L}evel vision tasks. The benchmark comprises 2,469 real-world degraded images covering 16 low-level degradation tasks, and 28,919 restored images produced by 10 state-of-the-art large-scale generative models and 21 conventional restoration models, which are annotated with 152,020 expert-level pairwise human preferences and 28,334 quality scores. Built upon LL-Bench, we present a systematic diagnosis that reveals the performance boundaries and unique failure modes of large-scale generative models across diverse low-level vision tasks, compared with conventional representative restoration approaches. Moreover, we investigate the effectiveness of current quality evaluation metrics on LL-Bench, which exhibit significant discrepancy with human ratings. To better align restored-image quality assessment with human preferences, we further propose \textbf{LL-Score}, an MLLM-based evaluator that captures both restoration quality and hallucination existence. Extensive experiments demonstrate that LL-score not only outperforms existing image quality assessment metrics, but also serves as a promising reward model for training generative models on low-level vision tasks. |
| title | LL-Bench: Rethinking Low-Level Vision Evaluation in the Era of Large-Scale Generative Models |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2606.02535 |