VLIC: Vision-Language Models As Perceptual Judges for Human-Aligned Image Compression
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909967835987968 |
|---|---|
| author | Sargent, Kyle Gao, Ruiqi Henzler, Philipp Herrmann, Charles Holynski, Aleksander Fei-Fei, Li Wu, Jiajun Zhang, Jason |
| author_facet | Sargent, Kyle Gao, Ruiqi Henzler, Philipp Herrmann, Charles Holynski, Aleksander Fei-Fei, Li Wu, Jiajun Zhang, Jason |
| contents | Evaluations of image compression performance which include human preferences have generally found that naive distortion functions such as MSE are insufficiently aligned to human perception. In order to align compression models to human perception, prior work has employed differentiable perceptual losses consisting of neural networks calibrated on large-scale datasets of human psycho-visual judgments. We show that, surprisingly, state-of-the-art vision-language models (VLMs) can replicate binary human two-alternative forced choice (2AFC) judgments zero-shot when asked to reason about the differences between pairs of images. Motivated to exploit the powerful zero-shot visual reasoning capabilities of VLMs, we propose Vision-Language Models for Image Compression (VLIC), a diffusion-based image compression system designed to be post-trained with binary VLM judgments. VLIC leverages existing techniques for diffusion model post-training with preferences, rather than distilling the VLM judgments into a separate perceptual loss network. We show that calibrating this system on VLM judgments produces competitive or state-of-the-art performance on human-aligned visual compression depending on the dataset, according to perceptual metrics and large-scale user studies. We additionally conduct an extensive analysis of the VLM-based reward design and training procedure and share important insights. More visuals are available at https://kylesargent.github.io/vlic |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_15701 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | VLIC: Vision-Language Models As Perceptual Judges for Human-Aligned Image Compression Sargent, Kyle Gao, Ruiqi Henzler, Philipp Herrmann, Charles Holynski, Aleksander Fei-Fei, Li Wu, Jiajun Zhang, Jason Computer Vision and Pattern Recognition Evaluations of image compression performance which include human preferences have generally found that naive distortion functions such as MSE are insufficiently aligned to human perception. In order to align compression models to human perception, prior work has employed differentiable perceptual losses consisting of neural networks calibrated on large-scale datasets of human psycho-visual judgments. We show that, surprisingly, state-of-the-art vision-language models (VLMs) can replicate binary human two-alternative forced choice (2AFC) judgments zero-shot when asked to reason about the differences between pairs of images. Motivated to exploit the powerful zero-shot visual reasoning capabilities of VLMs, we propose Vision-Language Models for Image Compression (VLIC), a diffusion-based image compression system designed to be post-trained with binary VLM judgments. VLIC leverages existing techniques for diffusion model post-training with preferences, rather than distilling the VLM judgments into a separate perceptual loss network. We show that calibrating this system on VLM judgments produces competitive or state-of-the-art performance on human-aligned visual compression depending on the dataset, according to perceptual metrics and large-scale user studies. We additionally conduct an extensive analysis of the VLM-based reward design and training procedure and share important insights. More visuals are available at https://kylesargent.github.io/vlic |
| title | VLIC: Vision-Language Models As Perceptual Judges for Human-Aligned Image Compression |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2512.15701 |