Beyond Grids: Exploring Elastic Input Sampling for Vision Transformers
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2023
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866918164286144512 |
|---|---|
| author | Pardyl, Adam Kurzejamski, Grzegorz Olszewski, Jan Trzciński, Tomasz Zieliński, Bartosz |
| author_facet | Pardyl, Adam Kurzejamski, Grzegorz Olszewski, Jan Trzciński, Tomasz Zieliński, Bartosz |
| contents | Vision transformers have excelled in various computer vision tasks but mostly rely on rigid input sampling using a fixed-size grid of patches. It limits their applicability in real-world problems, such as active visual exploration, where patches have various scales and positions. Our paper addresses this limitation by formalizing the concept of input elasticity for vision transformers and introducing an evaluation protocol for measuring this elasticity. Moreover, we propose modifications to the transformer architecture and training regime, which increase its elasticity. Through extensive experimentation, we spotlight opportunities and challenges associated with such architecture. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_13353 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Beyond Grids: Exploring Elastic Input Sampling for Vision Transformers Pardyl, Adam Kurzejamski, Grzegorz Olszewski, Jan Trzciński, Tomasz Zieliński, Bartosz Computer Vision and Pattern Recognition Vision transformers have excelled in various computer vision tasks but mostly rely on rigid input sampling using a fixed-size grid of patches. It limits their applicability in real-world problems, such as active visual exploration, where patches have various scales and positions. Our paper addresses this limitation by formalizing the concept of input elasticity for vision transformers and introducing an evaluation protocol for measuring this elasticity. Moreover, we propose modifications to the transformer architecture and training regime, which increase its elasticity. Through extensive experimentation, we spotlight opportunities and challenges associated with such architecture. |
| title | Beyond Grids: Exploring Elastic Input Sampling for Vision Transformers |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2309.13353 |