MAST: Video Polyp Segmentation with a Mixture-Attention Siamese Transformer
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910305949319168 |
|---|---|
| author | Chen, Geng Yang, Junqing Pu, Xiaozhou Ji, Ge-Peng Xiong, Huan Pan, Yongsheng Cui, Hengfei Xia, Yong |
| author_facet | Chen, Geng Yang, Junqing Pu, Xiaozhou Ji, Ge-Peng Xiong, Huan Pan, Yongsheng Cui, Hengfei Xia, Yong |
| contents | Accurate segmentation of polyps from colonoscopy videos is of great significance to polyp treatment and early prevention of colorectal cancer. However, it is challenging due to the difficulties associated with modelling long-range spatio-temporal relationships within a colonoscopy video. In this paper, we address this challenging task with a novel Mixture-Attention Siamese Transformer (MAST), which explicitly models the long-range spatio-temporal relationships with a mixture-attention mechanism for accurate polyp segmentation. Specifically, we first construct a Siamese transformer architecture to jointly encode paired video frames for their feature representations. We then design a mixture-attention module to exploit the intra-frame and inter-frame correlations, enhancing the features with rich spatio-temporal relationships. Finally, the enhanced features are fed to two parallel decoders for predicting the segmentation maps. To the best of our knowledge, our MAST is the first transformer model dedicated to video polyp segmentation. Extensive experiments on the large-scale SUN-SEG benchmark demonstrate the superior performance of MAST in comparison with the cutting-edge competitors. Our code is publicly available at https://github.com/Junqing-Yang/MAST. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_12439 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MAST: Video Polyp Segmentation with a Mixture-Attention Siamese Transformer Chen, Geng Yang, Junqing Pu, Xiaozhou Ji, Ge-Peng Xiong, Huan Pan, Yongsheng Cui, Hengfei Xia, Yong Computer Vision and Pattern Recognition Accurate segmentation of polyps from colonoscopy videos is of great significance to polyp treatment and early prevention of colorectal cancer. However, it is challenging due to the difficulties associated with modelling long-range spatio-temporal relationships within a colonoscopy video. In this paper, we address this challenging task with a novel Mixture-Attention Siamese Transformer (MAST), which explicitly models the long-range spatio-temporal relationships with a mixture-attention mechanism for accurate polyp segmentation. Specifically, we first construct a Siamese transformer architecture to jointly encode paired video frames for their feature representations. We then design a mixture-attention module to exploit the intra-frame and inter-frame correlations, enhancing the features with rich spatio-temporal relationships. Finally, the enhanced features are fed to two parallel decoders for predicting the segmentation maps. To the best of our knowledge, our MAST is the first transformer model dedicated to video polyp segmentation. Extensive experiments on the large-scale SUN-SEG benchmark demonstrate the superior performance of MAST in comparison with the cutting-edge competitors. Our code is publicly available at https://github.com/Junqing-Yang/MAST. |
| title | MAST: Video Polyp Segmentation with a Mixture-Attention Siamese Transformer |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2401.12439 |