Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2501.16481 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917904083058688 |
|---|---|
| author | Kamboj, Payal Banerjee, Ayan Xu, Bin Gupta, Sandeep |
| author_facet | Kamboj, Payal Banerjee, Ayan Xu, Bin Gupta, Sandeep |
| contents | Rare events, due to their infrequent occurrences, do not have much data, and hence deep learning techniques fail in estimating the distribution for such data. Open-vocabulary models represent an innovative approach to image classification. Unlike traditional models, these models classify images into any set of categories specified with natural language prompts during inference. These prompts usually comprise manually crafted templates (e.g., 'a photo of a {}') that are filled in with the names of each category. This paper introduces a simple yet effective method for generating highly accurate and contextually descriptive prompts containing discriminative characteristics. Rare event detection, especially in medicine, is more challenging due to low inter-class and high intra-class variability. To address these, we propose a novel approach that uses domain-specific expert knowledge on rare events to generate customized and contextually relevant prompts, which are then used by large language models for image classification. Our zero-shot, privacy-preserving method enhances rare event classification without additional training, outperforming state-of-the-art techniques. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_16481 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Generating customized prompts for Zero-Shot Rare Event Medical Image Classification using LLM Kamboj, Payal Banerjee, Ayan Xu, Bin Gupta, Sandeep Computer Vision and Pattern Recognition Rare events, due to their infrequent occurrences, do not have much data, and hence deep learning techniques fail in estimating the distribution for such data. Open-vocabulary models represent an innovative approach to image classification. Unlike traditional models, these models classify images into any set of categories specified with natural language prompts during inference. These prompts usually comprise manually crafted templates (e.g., 'a photo of a {}') that are filled in with the names of each category. This paper introduces a simple yet effective method for generating highly accurate and contextually descriptive prompts containing discriminative characteristics. Rare event detection, especially in medicine, is more challenging due to low inter-class and high intra-class variability. To address these, we propose a novel approach that uses domain-specific expert knowledge on rare events to generate customized and contextually relevant prompts, which are then used by large language models for image classification. Our zero-shot, privacy-preserving method enhances rare event classification without additional training, outperforming state-of-the-art techniques. |
| title | Generating customized prompts for Zero-Shot Rare Event Medical Image Classification using LLM |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2501.16481 |