A Rigorous Behavior Assessment of CNNs Using a Data-Domain Sampling Regime
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915507671662592 |
|---|---|
| author | Jiang, Shuning Chao, Wei-Lun Haehn, Daniel Pfister, Hanspeter Chen, Jian |
| author_facet | Jiang, Shuning Chao, Wei-Lun Haehn, Daniel Pfister, Hanspeter Chen, Jian |
| contents | We present a data-domain sampling regime for quantifying CNNs' graphic perception behaviors. This regime lets us evaluate CNNs' ratio estimation ability in bar charts from three perspectives: sensitivity to training-test distribution discrepancies, stability to limited samples, and relative expertise to human observers. After analyzing 16 million trials from 800 CNNs models and 6,825 trials from 113 human participants, we arrived at a simple and actionable conclusion: CNNs can outperform humans and their biases simply depend on the training-test distance. We show evidence of this simple, elegant behavior of the machines when they interpret visualization images. osf.io/gfqc3 provides registration, the code for our sampling regime, and experimental results. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_03866 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Rigorous Behavior Assessment of CNNs Using a Data-Domain Sampling Regime Jiang, Shuning Chao, Wei-Lun Haehn, Daniel Pfister, Hanspeter Chen, Jian Machine Learning Computer Vision and Pattern Recognition Human-Computer Interaction We present a data-domain sampling regime for quantifying CNNs' graphic perception behaviors. This regime lets us evaluate CNNs' ratio estimation ability in bar charts from three perspectives: sensitivity to training-test distribution discrepancies, stability to limited samples, and relative expertise to human observers. After analyzing 16 million trials from 800 CNNs models and 6,825 trials from 113 human participants, we arrived at a simple and actionable conclusion: CNNs can outperform humans and their biases simply depend on the training-test distance. We show evidence of this simple, elegant behavior of the machines when they interpret visualization images. osf.io/gfqc3 provides registration, the code for our sampling regime, and experimental results. |
| title | A Rigorous Behavior Assessment of CNNs Using a Data-Domain Sampling Regime |
| topic | Machine Learning Computer Vision and Pattern Recognition Human-Computer Interaction |
| url | https://arxiv.org/abs/2507.03866 |