VMamba: Visual State Space Model
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917880270946304 |
|---|---|
| author | Liu, Yue Tian, Yunjie Zhao, Yuzhong Yu, Hongtian Xie, Lingxi Wang, Yaowei Ye, Qixiang Jiao, Jianbin Liu, Yunfan |
| author_facet | Liu, Yue Tian, Yunjie Zhao, Yuzhong Yu, Hongtian Xie, Lingxi Wang, Yaowei Ye, Qixiang Jiao, Jianbin Liu, Yunfan |
| contents | Designing computationally efficient network architectures remains an ongoing necessity in computer vision. In this paper, we adapt Mamba, a state-space language model, into VMamba, a vision backbone with linear time complexity. At the core of VMamba is a stack of Visual State-Space (VSS) blocks with the 2D Selective Scan (SS2D) module. By traversing along four scanning routes, SS2D bridges the gap between the ordered nature of 1D selective scan and the non-sequential structure of 2D vision data, which facilitates the collection of contextual information from various sources and perspectives. Based on the VSS blocks, we develop a family of VMamba architectures and accelerate them through a succession of architectural and implementation enhancements. Extensive experiments demonstrate VMamba's promising performance across diverse visual perception tasks, highlighting its superior input scaling efficiency compared to existing benchmark models. Source code is available at https://github.com/MzeroMiko/VMamba. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_10166 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | VMamba: Visual State Space Model Liu, Yue Tian, Yunjie Zhao, Yuzhong Yu, Hongtian Xie, Lingxi Wang, Yaowei Ye, Qixiang Jiao, Jianbin Liu, Yunfan Computer Vision and Pattern Recognition Designing computationally efficient network architectures remains an ongoing necessity in computer vision. In this paper, we adapt Mamba, a state-space language model, into VMamba, a vision backbone with linear time complexity. At the core of VMamba is a stack of Visual State-Space (VSS) blocks with the 2D Selective Scan (SS2D) module. By traversing along four scanning routes, SS2D bridges the gap between the ordered nature of 1D selective scan and the non-sequential structure of 2D vision data, which facilitates the collection of contextual information from various sources and perspectives. Based on the VSS blocks, we develop a family of VMamba architectures and accelerate them through a succession of architectural and implementation enhancements. Extensive experiments demonstrate VMamba's promising performance across diverse visual perception tasks, highlighting its superior input scaling efficiency compared to existing benchmark models. Source code is available at https://github.com/MzeroMiko/VMamba. |
| title | VMamba: Visual State Space Model |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2401.10166 |