Prompt-Based Exemplar Super-Compression and Regeneration for Class-Incremental Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913966566932480 |
|---|---|
| author | Duan, Ruxiao Chen, Jieneng Kortylewski, Adam Yuille, Alan Liu, Yaoyao |
| author_facet | Duan, Ruxiao Chen, Jieneng Kortylewski, Adam Yuille, Alan Liu, Yaoyao |
| contents | Replay-based methods in class-incremental learning (CIL) have attained remarkable success. Despite their effectiveness, the inherent memory restriction results in saving a limited number of exemplars with poor diversity. In this paper, we introduce PESCR, a novel approach that substantially increases the quantity and enhances the diversity of exemplars based on a pre-trained general-purpose diffusion model, without fine-tuning it on target datasets or storing it in the memory buffer. Images are compressed into visual and textual prompts, which are saved instead of the original images, decreasing memory consumption by a factor of 24. In subsequent phases, diverse exemplars are regenerated by the diffusion model. We further propose partial compression and diffusion-based data augmentation to minimize the domain gap between generated exemplars and real images. PESCR significantly improves CIL performance across multiple benchmarks, e.g., 3.2% above the previous state-of-the-art on ImageNet-100. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_18266 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Prompt-Based Exemplar Super-Compression and Regeneration for Class-Incremental Learning Duan, Ruxiao Chen, Jieneng Kortylewski, Adam Yuille, Alan Liu, Yaoyao Computer Vision and Pattern Recognition Replay-based methods in class-incremental learning (CIL) have attained remarkable success. Despite their effectiveness, the inherent memory restriction results in saving a limited number of exemplars with poor diversity. In this paper, we introduce PESCR, a novel approach that substantially increases the quantity and enhances the diversity of exemplars based on a pre-trained general-purpose diffusion model, without fine-tuning it on target datasets or storing it in the memory buffer. Images are compressed into visual and textual prompts, which are saved instead of the original images, decreasing memory consumption by a factor of 24. In subsequent phases, diverse exemplars are regenerated by the diffusion model. We further propose partial compression and diffusion-based data augmentation to minimize the domain gap between generated exemplars and real images. PESCR significantly improves CIL performance across multiple benchmarks, e.g., 3.2% above the previous state-of-the-art on ImageNet-100. |
| title | Prompt-Based Exemplar Super-Compression and Regeneration for Class-Incremental Learning |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2311.18266 |