Restore Anything Model via Efficient Degradation Adaptation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913616883613696 |
|---|---|
| author | Ren, Bin Zamfir, Eduard Wu, Zongwei Li, Yawei Li, Yidi Paudel, Danda Pani Timofte, Radu Yang, Ming-Hsuan Sebe, Nicu |
| author_facet | Ren, Bin Zamfir, Eduard Wu, Zongwei Li, Yawei Li, Yidi Paudel, Danda Pani Timofte, Radu Yang, Ming-Hsuan Sebe, Nicu |
| contents | With the proliferation of mobile devices, the need for an efficient model to restore any degraded image has become increasingly significant and impactful. Traditional approaches typically involve training dedicated models for each specific degradation, resulting in inefficiency and redundancy. More recent solutions either introduce additional modules to learn visual prompts significantly increasing model size or incorporate cross-modal transfer from large language models trained on vast datasets, adding complexity to the system architecture. In contrast, our approach, termed RAM, takes a unified path that leverages inherent similarities across various degradations to enable both efficient and comprehensive restoration through a joint embedding mechanism without scaling up the model or relying on large multimodal models. Specifically, we examine the sub-latent space of each input, identifying key components and reweighting them in a gated manner. This intrinsic degradation awareness is further combined with contextualized attention in an X-shaped framework, enhancing local-global interactions. Extensive benchmarking in an all-in-one restoration setting confirms RAM's SOTA performance, reducing model complexity by approximately 82% in trainable parameters and 85% in FLOPs. Our code and models will be publicly available. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_13372 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Restore Anything Model via Efficient Degradation Adaptation Ren, Bin Zamfir, Eduard Wu, Zongwei Li, Yawei Li, Yidi Paudel, Danda Pani Timofte, Radu Yang, Ming-Hsuan Sebe, Nicu Computer Vision and Pattern Recognition With the proliferation of mobile devices, the need for an efficient model to restore any degraded image has become increasingly significant and impactful. Traditional approaches typically involve training dedicated models for each specific degradation, resulting in inefficiency and redundancy. More recent solutions either introduce additional modules to learn visual prompts significantly increasing model size or incorporate cross-modal transfer from large language models trained on vast datasets, adding complexity to the system architecture. In contrast, our approach, termed RAM, takes a unified path that leverages inherent similarities across various degradations to enable both efficient and comprehensive restoration through a joint embedding mechanism without scaling up the model or relying on large multimodal models. Specifically, we examine the sub-latent space of each input, identifying key components and reweighting them in a gated manner. This intrinsic degradation awareness is further combined with contextualized attention in an X-shaped framework, enhancing local-global interactions. Extensive benchmarking in an all-in-one restoration setting confirms RAM's SOTA performance, reducing model complexity by approximately 82% in trainable parameters and 85% in FLOPs. Our code and models will be publicly available. |
| title | Restore Anything Model via Efficient Degradation Adaptation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2407.13372 |