Automatic Prompt Generation and Grounding Object Detection for Zero-Shot Image Anomaly Detection
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913590179528704 |
|---|---|
| author | Cheung, Tsun-Hin Fung, Ka-Chun Lai, Songjiang Lin, Kwan-Ho Ng, Vincent Lam, Kin-Man |
| author_facet | Cheung, Tsun-Hin Fung, Ka-Chun Lai, Songjiang Lin, Kwan-Ho Ng, Vincent Lam, Kin-Man |
| contents | Identifying defects and anomalies in industrial products is a critical quality control task. Traditional manual inspection methods are slow, subjective, and error-prone. In this work, we propose a novel zero-shot training-free approach for automated industrial image anomaly detection using a multimodal machine learning pipeline, consisting of three foundation models. Our method first uses a large language model, i.e., GPT-3. generate text prompts describing the expected appearances of normal and abnormal products. We then use a grounding object detection model, called Grounding DINO, to locate the product in the image. Finally, we compare the cropped product image patches to the generated prompts using a zero-shot image-text matching model, called CLIP, to identify any anomalies. Our experiments on two datasets of industrial product images, namely MVTec-AD and VisA, demonstrate the effectiveness of this method, achieving high accuracy in detecting various types of defects and anomalies without the need for model training. Our proposed model enables efficient, scalable, and objective quality control in industrial manufacturing settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_19220 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Automatic Prompt Generation and Grounding Object Detection for Zero-Shot Image Anomaly Detection Cheung, Tsun-Hin Fung, Ka-Chun Lai, Songjiang Lin, Kwan-Ho Ng, Vincent Lam, Kin-Man Computer Vision and Pattern Recognition Multimedia Identifying defects and anomalies in industrial products is a critical quality control task. Traditional manual inspection methods are slow, subjective, and error-prone. In this work, we propose a novel zero-shot training-free approach for automated industrial image anomaly detection using a multimodal machine learning pipeline, consisting of three foundation models. Our method first uses a large language model, i.e., GPT-3. generate text prompts describing the expected appearances of normal and abnormal products. We then use a grounding object detection model, called Grounding DINO, to locate the product in the image. Finally, we compare the cropped product image patches to the generated prompts using a zero-shot image-text matching model, called CLIP, to identify any anomalies. Our experiments on two datasets of industrial product images, namely MVTec-AD and VisA, demonstrate the effectiveness of this method, achieving high accuracy in detecting various types of defects and anomalies without the need for model training. Our proposed model enables efficient, scalable, and objective quality control in industrial manufacturing settings. |
| title | Automatic Prompt Generation and Grounding Object Detection for Zero-Shot Image Anomaly Detection |
| topic | Computer Vision and Pattern Recognition Multimedia |
| url | https://arxiv.org/abs/2411.19220 |