Memory-Modular Classification: Learning to Generalize with Memory Replacement
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909571737452544 |
|---|---|
| author | Kang, Dahyun Iscen, Ahmet Jo, Eunchan Choi, Sua Cho, Minsu Schmid, Cordelia |
| author_facet | Kang, Dahyun Iscen, Ahmet Jo, Eunchan Choi, Sua Cho, Minsu Schmid, Cordelia |
| contents | We propose a novel memory-modular learner for image classification that separates knowledge memorization from reasoning. Our model enables effective generalization to new classes by simply replacing the memory contents, without the need for model retraining. Unlike traditional models that encode both world knowledge and task-specific skills into their weights during training, our model stores knowledge in the external memory of web-crawled image and text data. At inference time, the model dynamically selects relevant content from the memory based on the input image, allowing it to adapt to arbitrary classes by simply replacing the memory contents. The key differentiator that our learner meta-learns to perform classification tasks with noisy web data from unseen classes, resulting in robust performance across various classification scenarios. Experimental results demonstrate the promising performance and versatility of our approach in handling diverse classification tasks, including zero-shot/few-shot classification of unseen classes, fine-grained classification, and class-incremental classification. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_06021 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Memory-Modular Classification: Learning to Generalize with Memory Replacement Kang, Dahyun Iscen, Ahmet Jo, Eunchan Choi, Sua Cho, Minsu Schmid, Cordelia Computer Vision and Pattern Recognition We propose a novel memory-modular learner for image classification that separates knowledge memorization from reasoning. Our model enables effective generalization to new classes by simply replacing the memory contents, without the need for model retraining. Unlike traditional models that encode both world knowledge and task-specific skills into their weights during training, our model stores knowledge in the external memory of web-crawled image and text data. At inference time, the model dynamically selects relevant content from the memory based on the input image, allowing it to adapt to arbitrary classes by simply replacing the memory contents. The key differentiator that our learner meta-learns to perform classification tasks with noisy web data from unseen classes, resulting in robust performance across various classification scenarios. Experimental results demonstrate the promising performance and versatility of our approach in handling diverse classification tasks, including zero-shot/few-shot classification of unseen classes, fine-grained classification, and class-incremental classification. |
| title | Memory-Modular Classification: Learning to Generalize with Memory Replacement |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2504.06021 |