CAMSIC: Content-aware Masked Image Modeling Transformer for Stereo Image Compression
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866929702842662912 |
|---|---|
| author | Zhang, Xinjie Gao, Shenyuan Liu, Zhening Shao, Jiawei Ge, Xingtong He, Dailan Xu, Tongda Wang, Yan Zhang, Jun |
| author_facet | Zhang, Xinjie Gao, Shenyuan Liu, Zhening Shao, Jiawei Ge, Xingtong He, Dailan Xu, Tongda Wang, Yan Zhang, Jun |
| contents | Existing learning-based stereo image codec adopt sophisticated transformation with simple entropy models derived from single image codecs to encode latent representations. However, those entropy models struggle to effectively capture the spatial-disparity characteristics inherent in stereo images, which leads to suboptimal rate-distortion results. In this paper, we propose a stereo image compression framework, named CAMSIC. CAMSIC independently transforms each image to latent representation and employs a powerful decoder-free Transformer entropy model to capture both spatial and disparity dependencies, by introducing a novel content-aware masked image modeling (MIM) technique. Our content-aware MIM facilitates efficient bidirectional interaction between prior information and estimated tokens, which naturally obviates the need for an extra Transformer decoder. Experiments show that our stereo image codec achieves state-of-the-art rate-distortion performance on two stereo image datasets Cityscapes and InStereo2K with fast encoding and decoding speed. Code is available at https://github.com/Xinjie-Q/CAMSIC. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_08505 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | CAMSIC: Content-aware Masked Image Modeling Transformer for Stereo Image Compression Zhang, Xinjie Gao, Shenyuan Liu, Zhening Shao, Jiawei Ge, Xingtong He, Dailan Xu, Tongda Wang, Yan Zhang, Jun Image and Video Processing Artificial Intelligence Computer Vision and Pattern Recognition Multimedia Existing learning-based stereo image codec adopt sophisticated transformation with simple entropy models derived from single image codecs to encode latent representations. However, those entropy models struggle to effectively capture the spatial-disparity characteristics inherent in stereo images, which leads to suboptimal rate-distortion results. In this paper, we propose a stereo image compression framework, named CAMSIC. CAMSIC independently transforms each image to latent representation and employs a powerful decoder-free Transformer entropy model to capture both spatial and disparity dependencies, by introducing a novel content-aware masked image modeling (MIM) technique. Our content-aware MIM facilitates efficient bidirectional interaction between prior information and estimated tokens, which naturally obviates the need for an extra Transformer decoder. Experiments show that our stereo image codec achieves state-of-the-art rate-distortion performance on two stereo image datasets Cityscapes and InStereo2K with fast encoding and decoding speed. Code is available at https://github.com/Xinjie-Q/CAMSIC. |
| title | CAMSIC: Content-aware Masked Image Modeling Transformer for Stereo Image Compression |
| topic | Image and Video Processing Artificial Intelligence Computer Vision and Pattern Recognition Multimedia |
| url | https://arxiv.org/abs/2403.08505 |