A Low-Resolution Image is Worth 1x1 Words: Enabling Fine Image Super-Resolution with Transformers and TaylorShift
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917053915463680 |
|---|---|
| author | Nagaraju, Sanath Budakegowdanadoddi Moser, Brian Bernhard Nauen, Tobias Christian Frolov, Stanislav Raue, Federico Dengel, Andreas |
| author_facet | Nagaraju, Sanath Budakegowdanadoddi Moser, Brian Bernhard Nauen, Tobias Christian Frolov, Stanislav Raue, Federico Dengel, Andreas |
| contents | Transformer-based architectures have recently advanced the image reconstruction quality of super-resolution (SR) models. Yet, their scalability remains limited by quadratic attention costs and coarse patch embeddings that weaken pixel-level fidelity. We propose TaylorIR, a plug-and-play framework that enforces 1x1 patch embeddings for true pixel-wise reasoning and replaces conventional self-attention with TaylorShift, a Taylor-series-based attention mechanism enabling full token interactions with near-linear complexity. Across multiple SR benchmarks, TaylorIR delivers state-of-the-art performance while reducing memory consumption by up to 60%, effectively bridging the gap between fine-grained detail restoration and efficient transformer scaling. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_10231 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Low-Resolution Image is Worth 1x1 Words: Enabling Fine Image Super-Resolution with Transformers and TaylorShift Nagaraju, Sanath Budakegowdanadoddi Moser, Brian Bernhard Nauen, Tobias Christian Frolov, Stanislav Raue, Federico Dengel, Andreas Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning Multimedia Transformer-based architectures have recently advanced the image reconstruction quality of super-resolution (SR) models. Yet, their scalability remains limited by quadratic attention costs and coarse patch embeddings that weaken pixel-level fidelity. We propose TaylorIR, a plug-and-play framework that enforces 1x1 patch embeddings for true pixel-wise reasoning and replaces conventional self-attention with TaylorShift, a Taylor-series-based attention mechanism enabling full token interactions with near-linear complexity. Across multiple SR benchmarks, TaylorIR delivers state-of-the-art performance while reducing memory consumption by up to 60%, effectively bridging the gap between fine-grained detail restoration and efficient transformer scaling. |
| title | A Low-Resolution Image is Worth 1x1 Words: Enabling Fine Image Super-Resolution with Transformers and TaylorShift |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning Multimedia |
| url | https://arxiv.org/abs/2411.10231 |