Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2407.19001 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913073095245824 |
|---|---|
| author | Cendra, Fernando Julio Li, Xinghui Han, Kai |
| author_facet | Cendra, Fernando Julio Li, Xinghui Han, Kai |
| contents | This paper studies effective prompt pool learning for Continual Category Discovery (CCD), a challenging open-world setting where a model must discover novel categories from a continuous stream of unlabelled data containing both known and novel classes, while mitigating catastrophic forgetting of previously learned concepts. We introduce a series of novel prompt-pool-based frameworks for CCD, each exploring a different design of prompt pools. First, we propose PromptCCD, which focuses on global class prototypes via a Gaussian Mixture Prompt (GMP) module. GMP fits a generative Gaussian mixture model over feature embeddings, where each mixture component serves as both a class prototype and a dynamic prompt that conditions the backbone's representations. This design enables label-free prompt selection and on-the-fly estimation of the number of emerging categories. Through a systematic spectrum study, we then show that category count, rather than sample size, is the primary bottleneck for discovery performance, motivating the need for finer-grained representations. Building on this finding, we propose PromptCCD++, which focuses on object-part prototypes via Part-level Prompting (PLP) modules. PLP decomposes prompt pool into multiple, specialized part-level prompt pools. During discovery phase, these pools dynamically assign part-specific prompts to local object regions without the need for manual part annotations, enabling the model to learn object-part representations that boost category discovery. Extensive evaluations on both generic and fine-grained benchmarks, supported by comprehensive ablation studies, demonstrate the effectiveness of our framework for CCD. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_19001 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Effective Prompt Pool Learning for Continual Category Discovery Cendra, Fernando Julio Li, Xinghui Han, Kai Computer Vision and Pattern Recognition This paper studies effective prompt pool learning for Continual Category Discovery (CCD), a challenging open-world setting where a model must discover novel categories from a continuous stream of unlabelled data containing both known and novel classes, while mitigating catastrophic forgetting of previously learned concepts. We introduce a series of novel prompt-pool-based frameworks for CCD, each exploring a different design of prompt pools. First, we propose PromptCCD, which focuses on global class prototypes via a Gaussian Mixture Prompt (GMP) module. GMP fits a generative Gaussian mixture model over feature embeddings, where each mixture component serves as both a class prototype and a dynamic prompt that conditions the backbone's representations. This design enables label-free prompt selection and on-the-fly estimation of the number of emerging categories. Through a systematic spectrum study, we then show that category count, rather than sample size, is the primary bottleneck for discovery performance, motivating the need for finer-grained representations. Building on this finding, we propose PromptCCD++, which focuses on object-part prototypes via Part-level Prompting (PLP) modules. PLP decomposes prompt pool into multiple, specialized part-level prompt pools. During discovery phase, these pools dynamically assign part-specific prompts to local object regions without the need for manual part annotations, enabling the model to learn object-part representations that boost category discovery. Extensive evaluations on both generic and fine-grained benchmarks, supported by comprehensive ablation studies, demonstrate the effectiveness of our framework for CCD. |
| title | Effective Prompt Pool Learning for Continual Category Discovery |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2407.19001 |