Hybrid Mamba-Transformer Decoder for Error-Correcting Codes
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915301203902464 |
|---|---|
| author | Cohen, Shy-el Choukroun, Yoni Nachmani, Eliya |
| author_facet | Cohen, Shy-el Choukroun, Yoni Nachmani, Eliya |
| contents | We introduce a novel deep learning method for decoding error correction codes based on the Mamba architecture, enhanced with Transformer layers. Our approach proposes a hybrid decoder that leverages Mamba's efficient sequential modeling while maintaining the global context capabilities of Transformers. To further improve performance, we design a novel layer-wise masking strategy applied to each Mamba layer, allowing selective attention to relevant code features at different depths. Additionally, we introduce a progressive layer-wise loss, supervising the network at intermediate stages and promoting robust feature extraction throughout the decoding process. Comprehensive experiments across a range of linear codes demonstrate that our method significantly outperforms Transformer-only decoders and standard Mamba models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_17834 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Hybrid Mamba-Transformer Decoder for Error-Correcting Codes Cohen, Shy-el Choukroun, Yoni Nachmani, Eliya Information Theory Artificial Intelligence Machine Learning We introduce a novel deep learning method for decoding error correction codes based on the Mamba architecture, enhanced with Transformer layers. Our approach proposes a hybrid decoder that leverages Mamba's efficient sequential modeling while maintaining the global context capabilities of Transformers. To further improve performance, we design a novel layer-wise masking strategy applied to each Mamba layer, allowing selective attention to relevant code features at different depths. Additionally, we introduce a progressive layer-wise loss, supervising the network at intermediate stages and promoting robust feature extraction throughout the decoding process. Comprehensive experiments across a range of linear codes demonstrate that our method significantly outperforms Transformer-only decoders and standard Mamba models. |
| title | Hybrid Mamba-Transformer Decoder for Error-Correcting Codes |
| topic | Information Theory Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2505.17834 |