Key-Graph Transformer for Image Restoration
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916116931018752 |
|---|---|
| author | Ren, Bin Li, Yawei Liang, Jingyun Ranjan, Rakesh Liu, Mengyuan Cucchiara, Rita Van Gool, Luc Sebe, Nicu |
| author_facet | Ren, Bin Li, Yawei Liang, Jingyun Ranjan, Rakesh Liu, Mengyuan Cucchiara, Rita Van Gool, Luc Sebe, Nicu |
| contents | While it is crucial to capture global information for effective image restoration (IR), integrating such cues into transformer-based methods becomes computationally expensive, especially with high input resolution. Furthermore, the self-attention mechanism in transformers is prone to considering unnecessary global cues from unrelated objects or regions, introducing computational inefficiencies. In response to these challenges, we introduce the Key-Graph Transformer (KGT) in this paper. Specifically, KGT views patch features as graph nodes. The proposed Key-Graph Constructor efficiently forms a sparse yet representative Key-Graph by selectively connecting essential nodes instead of all the nodes. Then the proposed Key-Graph Attention is conducted under the guidance of the Key-Graph only among selected nodes with linear computational complexity within each window. Extensive experiments across 6 IR tasks confirm the proposed KGT's state-of-the-art performance, showcasing advancements both quantitatively and qualitatively. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_02634 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Key-Graph Transformer for Image Restoration Ren, Bin Li, Yawei Liang, Jingyun Ranjan, Rakesh Liu, Mengyuan Cucchiara, Rita Van Gool, Luc Sebe, Nicu Computer Vision and Pattern Recognition Machine Learning Image and Video Processing While it is crucial to capture global information for effective image restoration (IR), integrating such cues into transformer-based methods becomes computationally expensive, especially with high input resolution. Furthermore, the self-attention mechanism in transformers is prone to considering unnecessary global cues from unrelated objects or regions, introducing computational inefficiencies. In response to these challenges, we introduce the Key-Graph Transformer (KGT) in this paper. Specifically, KGT views patch features as graph nodes. The proposed Key-Graph Constructor efficiently forms a sparse yet representative Key-Graph by selectively connecting essential nodes instead of all the nodes. Then the proposed Key-Graph Attention is conducted under the guidance of the Key-Graph only among selected nodes with linear computational complexity within each window. Extensive experiments across 6 IR tasks confirm the proposed KGT's state-of-the-art performance, showcasing advancements both quantitatively and qualitatively. |
| title | Key-Graph Transformer for Image Restoration |
| topic | Computer Vision and Pattern Recognition Machine Learning Image and Video Processing |
| url | https://arxiv.org/abs/2402.02634 |