Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2310.08182 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909173116043264 |
|---|---|
| author | Li, Qiang Zhang, Dan Lei, Shengzhao Zhao, Xun Kamnoedboon, Porawit Li, WeiWei Dong, Junhao Li, Shuyan |
| author_facet | Li, Qiang Zhang, Dan Lei, Shengzhao Zhao, Xun Kamnoedboon, Porawit Li, WeiWei Dong, Junhao Li, Shuyan |
| contents | Despite the promising performance of existing visual models on public benchmarks, the critical assessment of their robustness for real-world applications remains an ongoing challenge. To bridge this gap, we propose an explainable visual dataset, XIMAGENET-12, to evaluate the robustness of visual models. XIMAGENET-12 consists of over 200K images with 15,410 manual semantic annotations. Specifically, we deliberately selected 12 categories from ImageNet, representing objects commonly encountered in practical life. To simulate real-world situations, we incorporated six diverse scenarios, such as overexposure, blurring, and color changes, etc. We further develop a quantitative criterion for robustness assessment, allowing for a nuanced understanding of how visual models perform under varying conditions, notably in relation to the background. We make the XIMAGENET-12 dataset and its corresponding code openly accessible at \url{https://sites.google.com/view/ximagenet-12/home}. We expect the introduction of the XIMAGENET-12 dataset will empower researchers to thoroughly evaluate the robustness of their visual models under challenging conditions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_08182 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | XIMAGENET-12: An Explainable AI Benchmark Dataset for Model Robustness Evaluation Li, Qiang Zhang, Dan Lei, Shengzhao Zhao, Xun Kamnoedboon, Porawit Li, WeiWei Dong, Junhao Li, Shuyan Computer Vision and Pattern Recognition Machine Learning Despite the promising performance of existing visual models on public benchmarks, the critical assessment of their robustness for real-world applications remains an ongoing challenge. To bridge this gap, we propose an explainable visual dataset, XIMAGENET-12, to evaluate the robustness of visual models. XIMAGENET-12 consists of over 200K images with 15,410 manual semantic annotations. Specifically, we deliberately selected 12 categories from ImageNet, representing objects commonly encountered in practical life. To simulate real-world situations, we incorporated six diverse scenarios, such as overexposure, blurring, and color changes, etc. We further develop a quantitative criterion for robustness assessment, allowing for a nuanced understanding of how visual models perform under varying conditions, notably in relation to the background. We make the XIMAGENET-12 dataset and its corresponding code openly accessible at \url{https://sites.google.com/view/ximagenet-12/home}. We expect the introduction of the XIMAGENET-12 dataset will empower researchers to thoroughly evaluate the robustness of their visual models under challenging conditions. |
| title | XIMAGENET-12: An Explainable AI Benchmark Dataset for Model Robustness Evaluation |
| topic | Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2310.08182 |