Open-World Dynamic Prompt and Continual Visual Representation Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917789027008512 |
|---|---|
| author | Kim, Youngeun Fang, Jun Zhang, Qin Cai, Zhaowei Shen, Yantao Duggal, Rahul Raychaudhuri, Dripta S. Tu, Zhuowen Xing, Yifan Dabeer, Onkar |
| author_facet | Kim, Youngeun Fang, Jun Zhang, Qin Cai, Zhaowei Shen, Yantao Duggal, Rahul Raychaudhuri, Dripta S. Tu, Zhuowen Xing, Yifan Dabeer, Onkar |
| contents | The open world is inherently dynamic, characterized by ever-evolving concepts and distributions. Continual learning (CL) in this dynamic open-world environment presents a significant challenge in effectively generalizing to unseen test-time classes. To address this challenge, we introduce a new practical CL setting tailored for open-world visual representation learning. In this setting, subsequent data streams systematically introduce novel classes that are disjoint from those seen in previous training phases, while also remaining distinct from the unseen test classes. In response, we present Dynamic Prompt and Representation Learner (DPaRL), a simple yet effective Prompt-based CL (PCL) method. Our DPaRL learns to generate dynamic prompts for inference, as opposed to relying on a static prompt pool in previous PCL methods. In addition, DPaRL jointly learns dynamic prompt generation and discriminative representation at each training stage whereas prior PCL methods only refine the prompt learning throughout the process. Our experimental results demonstrate the superiority of our approach, surpassing state-of-the-art methods on well-established open-world image retrieval benchmarks by an average of 4.7% improvement in Recall@1 performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_05312 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Open-World Dynamic Prompt and Continual Visual Representation Learning Kim, Youngeun Fang, Jun Zhang, Qin Cai, Zhaowei Shen, Yantao Duggal, Rahul Raychaudhuri, Dripta S. Tu, Zhuowen Xing, Yifan Dabeer, Onkar Computer Vision and Pattern Recognition The open world is inherently dynamic, characterized by ever-evolving concepts and distributions. Continual learning (CL) in this dynamic open-world environment presents a significant challenge in effectively generalizing to unseen test-time classes. To address this challenge, we introduce a new practical CL setting tailored for open-world visual representation learning. In this setting, subsequent data streams systematically introduce novel classes that are disjoint from those seen in previous training phases, while also remaining distinct from the unseen test classes. In response, we present Dynamic Prompt and Representation Learner (DPaRL), a simple yet effective Prompt-based CL (PCL) method. Our DPaRL learns to generate dynamic prompts for inference, as opposed to relying on a static prompt pool in previous PCL methods. In addition, DPaRL jointly learns dynamic prompt generation and discriminative representation at each training stage whereas prior PCL methods only refine the prompt learning throughout the process. Our experimental results demonstrate the superiority of our approach, surpassing state-of-the-art methods on well-established open-world image retrieval benchmarks by an average of 4.7% improvement in Recall@1 performance. |
| title | Open-World Dynamic Prompt and Continual Visual Representation Learning |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2409.05312 |