Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2402.18128 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908277215854592 |
|---|---|
| author | Guo, Han Hosseini, Ramtin Zhang, Ruiyi Somayajula, Sai Ashish Chowdhury, Ranak Roy Gupta, Rajesh K. Xie, Pengtao |
| author_facet | Guo, Han Hosseini, Ramtin Zhang, Ruiyi Somayajula, Sai Ashish Chowdhury, Ranak Roy Gupta, Rajesh K. Xie, Pengtao |
| contents | Masked Autoencoder (MAE) is a notable method for self-supervised pretraining in visual representation learning. It operates by randomly masking image patches and reconstructing these masked patches using the unmasked ones. A key limitation of MAE lies in its disregard for the varying informativeness of different patches, as it uniformly selects patches to mask. To overcome this, some approaches propose masking based on patch informativeness. However, these methods often do not consider the specific requirements of downstream tasks, potentially leading to suboptimal representations for these tasks. In response, we introduce the Multi-level Optimized Mask Autoencoder (MLO-MAE), a novel framework that leverages end-to-end feedback from downstream tasks to learn an optimal masking strategy during pretraining. Our experimental findings highlight MLO-MAE's significant advancements in visual representation learning. Compared to existing methods, it demonstrates remarkable improvements across diverse datasets and tasks, showcasing its adaptability and efficiency. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_18128 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Downstream Task Guided Masking Learning in Masked Autoencoders Using Multi-Level Optimization Guo, Han Hosseini, Ramtin Zhang, Ruiyi Somayajula, Sai Ashish Chowdhury, Ranak Roy Gupta, Rajesh K. Xie, Pengtao Computer Vision and Pattern Recognition Machine Learning Masked Autoencoder (MAE) is a notable method for self-supervised pretraining in visual representation learning. It operates by randomly masking image patches and reconstructing these masked patches using the unmasked ones. A key limitation of MAE lies in its disregard for the varying informativeness of different patches, as it uniformly selects patches to mask. To overcome this, some approaches propose masking based on patch informativeness. However, these methods often do not consider the specific requirements of downstream tasks, potentially leading to suboptimal representations for these tasks. In response, we introduce the Multi-level Optimized Mask Autoencoder (MLO-MAE), a novel framework that leverages end-to-end feedback from downstream tasks to learn an optimal masking strategy during pretraining. Our experimental findings highlight MLO-MAE's significant advancements in visual representation learning. Compared to existing methods, it demonstrates remarkable improvements across diverse datasets and tasks, showcasing its adaptability and efficiency. |
| title | Downstream Task Guided Masking Learning in Masked Autoencoders Using Multi-Level Optimization |
| topic | Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2402.18128 |