Do Vision Transformers See Like Humans? Evaluating their Perceptual Alignment
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916896382648320 |
|---|---|
| author | Hernández-Cámara, Pablo Jaén-Lorites, Jose Manuel Vila-Tomás, Jorge Laparra, Valero Malo, Jesus |
| author_facet | Hernández-Cámara, Pablo Jaén-Lorites, Jose Manuel Vila-Tomás, Jorge Laparra, Valero Malo, Jesus |
| contents | Vision Transformers (ViTs) achieve remarkable performance in image recognition tasks, yet their alignment with human perception remains largely unexplored. This study systematically analyzes how model size, dataset size, data augmentation and regularization impact ViT perceptual alignment with human judgments on the TID2013 dataset. Our findings confirm that larger models exhibit lower perceptual alignment, consistent with previous works. Increasing dataset diversity has a minimal impact, but exposing models to the same images more times reduces alignment. Stronger data augmentation and regularization further decrease alignment, especially in models exposed to repeated training cycles. These results highlight a trade-off between model complexity, training strategies, and alignment with human perception, raising important considerations for applications requiring human-like visual understanding. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_09850 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Do Vision Transformers See Like Humans? Evaluating their Perceptual Alignment Hernández-Cámara, Pablo Jaén-Lorites, Jose Manuel Vila-Tomás, Jorge Laparra, Valero Malo, Jesus Computer Vision and Pattern Recognition Vision Transformers (ViTs) achieve remarkable performance in image recognition tasks, yet their alignment with human perception remains largely unexplored. This study systematically analyzes how model size, dataset size, data augmentation and regularization impact ViT perceptual alignment with human judgments on the TID2013 dataset. Our findings confirm that larger models exhibit lower perceptual alignment, consistent with previous works. Increasing dataset diversity has a minimal impact, but exposing models to the same images more times reduces alignment. Stronger data augmentation and regularization further decrease alignment, especially in models exposed to repeated training cycles. These results highlight a trade-off between model complexity, training strategies, and alignment with human perception, raising important considerations for applications requiring human-like visual understanding. |
| title | Do Vision Transformers See Like Humans? Evaluating their Perceptual Alignment |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2508.09850 |