Restore Anything with Masks: Leveraging Mask Image Modeling for Blind All-in-One Image Restoration
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912049711284224 |
|---|---|
| author | Qin, Chu-Jie Wu, Rui-Qi Liu, Zikun Lin, Xin Guo, Chun-Le Park, Hyun Hee Li, Chongyi |
| author_facet | Qin, Chu-Jie Wu, Rui-Qi Liu, Zikun Lin, Xin Guo, Chun-Le Park, Hyun Hee Li, Chongyi |
| contents | All-in-one image restoration aims to handle multiple degradation types using one model. This paper proposes a simple pipeline for all-in-one blind image restoration to Restore Anything with Masks (RAM). We focus on the image content by utilizing Mask Image Modeling to extract intrinsic image information rather than distinguishing degradation types like other methods. Our pipeline consists of two stages: masked image pre-training and fine-tuning with mask attribute conductance. We design a straightforward masking pre-training approach specifically tailored for all-in-one image restoration. This approach enhances networks to prioritize the extraction of image content priors from various degradations, resulting in a more balanced performance across different restoration tasks and achieving stronger overall results. To bridge the gap of input integrity while preserving learned image priors as much as possible, we selectively fine-tuned a small portion of the layers. Specifically, the importance of each layer is ranked by the proposed Mask Attribute Conductance (MAC), and the layers with higher contributions are selected for finetuning. Extensive experiments demonstrate that our method achieves state-of-the-art performance. Our code and model will be released at \href{https://github.com/Dragonisss/RAM}{https://github.com/Dragonisss/RAM}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_19403 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Restore Anything with Masks: Leveraging Mask Image Modeling for Blind All-in-One Image Restoration Qin, Chu-Jie Wu, Rui-Qi Liu, Zikun Lin, Xin Guo, Chun-Le Park, Hyun Hee Li, Chongyi Computer Vision and Pattern Recognition All-in-one image restoration aims to handle multiple degradation types using one model. This paper proposes a simple pipeline for all-in-one blind image restoration to Restore Anything with Masks (RAM). We focus on the image content by utilizing Mask Image Modeling to extract intrinsic image information rather than distinguishing degradation types like other methods. Our pipeline consists of two stages: masked image pre-training and fine-tuning with mask attribute conductance. We design a straightforward masking pre-training approach specifically tailored for all-in-one image restoration. This approach enhances networks to prioritize the extraction of image content priors from various degradations, resulting in a more balanced performance across different restoration tasks and achieving stronger overall results. To bridge the gap of input integrity while preserving learned image priors as much as possible, we selectively fine-tuned a small portion of the layers. Specifically, the importance of each layer is ranked by the proposed Mask Attribute Conductance (MAC), and the layers with higher contributions are selected for finetuning. Extensive experiments demonstrate that our method achieves state-of-the-art performance. Our code and model will be released at \href{https://github.com/Dragonisss/RAM}{https://github.com/Dragonisss/RAM}. |
| title | Restore Anything with Masks: Leveraging Mask Image Modeling for Blind All-in-One Image Restoration |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2409.19403 |