CLIP-AD: A Language-Guided Staged Dual-Path Model for Zero-shot Anomaly Detection
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913251286056960 |
|---|---|
| author | Chen, Xuhai Zhang, Jiangning Tian, Guanzhong He, Haoyang Zhang, Wuhao Wang, Yabiao Wang, Chengjie Liu, Yong |
| author_facet | Chen, Xuhai Zhang, Jiangning Tian, Guanzhong He, Haoyang Zhang, Wuhao Wang, Yabiao Wang, Chengjie Liu, Yong |
| contents | This paper considers zero-shot Anomaly Detection (AD), performing AD without reference images of the test objects. We propose a framework called CLIP-AD to leverage the zero-shot capabilities of the large vision-language model CLIP. Firstly, we reinterpret the text prompts design from a distributional perspective and propose a Representative Vector Selection (RVS) paradigm to obtain improved text features. Secondly, we note opposite predictions and irrelevant highlights in the direct computation of the anomaly maps. To address these issues, we introduce a Staged Dual-Path model (SDP) that leverages features from various levels and applies architecture and feature surgery. Lastly, delving deeply into the two phenomena, we point out that the image and text features are not aligned in the joint embedding space. Thus, we introduce a fine-tuning strategy by adding linear layers and construct an extended model SDP+, further enhancing the performance. Abundant experiments demonstrate the effectiveness of our approach, e.g., on MVTec-AD, SDP outperforms the SOTA WinCLIP by +4.2/+10.7 in segmentation metrics F1-max/PRO, while SDP+ achieves +8.3/+20.5 improvements. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_00453 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | CLIP-AD: A Language-Guided Staged Dual-Path Model for Zero-shot Anomaly Detection Chen, Xuhai Zhang, Jiangning Tian, Guanzhong He, Haoyang Zhang, Wuhao Wang, Yabiao Wang, Chengjie Liu, Yong Computer Vision and Pattern Recognition This paper considers zero-shot Anomaly Detection (AD), performing AD without reference images of the test objects. We propose a framework called CLIP-AD to leverage the zero-shot capabilities of the large vision-language model CLIP. Firstly, we reinterpret the text prompts design from a distributional perspective and propose a Representative Vector Selection (RVS) paradigm to obtain improved text features. Secondly, we note opposite predictions and irrelevant highlights in the direct computation of the anomaly maps. To address these issues, we introduce a Staged Dual-Path model (SDP) that leverages features from various levels and applies architecture and feature surgery. Lastly, delving deeply into the two phenomena, we point out that the image and text features are not aligned in the joint embedding space. Thus, we introduce a fine-tuning strategy by adding linear layers and construct an extended model SDP+, further enhancing the performance. Abundant experiments demonstrate the effectiveness of our approach, e.g., on MVTec-AD, SDP outperforms the SOTA WinCLIP by +4.2/+10.7 in segmentation metrics F1-max/PRO, while SDP+ achieves +8.3/+20.5 improvements. |
| title | CLIP-AD: A Language-Guided Staged Dual-Path Model for Zero-shot Anomaly Detection |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2311.00453 |