Self-Supervised Radio Pre-training: Toward Foundational Models for Spectrogram Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917838702247936 |
|---|---|
| author | Aboulfotouh, Ahmed Eshaghbeigi, Ashkan Karslidis, Dimitrios Abou-Zeid, Hatem |
| author_facet | Aboulfotouh, Ahmed Eshaghbeigi, Ashkan Karslidis, Dimitrios Abou-Zeid, Hatem |
| contents | Foundational deep learning (DL) models are general models, trained on large, diverse, and unlabelled datasets, typically using self-supervised learning techniques have led to significant advancements especially in natural language processing. These pretrained models can be fine-tuned for related downstream tasks, offering faster development and reduced training costs, while often achieving improved performance. In this work, we introduce Masked Spectrogram Modeling, a novel self-supervised learning approach for pretraining foundational DL models on radio signals. Adopting a Convolutional LSTM architecture for efficient spatio-temporal processing, we pretrain the model with an unlabelled radio dataset collected from over-the-air measurements. Subsequently, the pretrained model is fine-tuned for two downstream tasks: spectrum forecasting and segmentation. Experimental results demonstrate that our methodology achieves competitive performance in both forecasting accuracy and segmentation, validating its effectiveness for developing foundational radio models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_09849 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Self-Supervised Radio Pre-training: Toward Foundational Models for Spectrogram Learning Aboulfotouh, Ahmed Eshaghbeigi, Ashkan Karslidis, Dimitrios Abou-Zeid, Hatem Signal Processing Artificial Intelligence Machine Learning Networking and Internet Architecture Foundational deep learning (DL) models are general models, trained on large, diverse, and unlabelled datasets, typically using self-supervised learning techniques have led to significant advancements especially in natural language processing. These pretrained models can be fine-tuned for related downstream tasks, offering faster development and reduced training costs, while often achieving improved performance. In this work, we introduce Masked Spectrogram Modeling, a novel self-supervised learning approach for pretraining foundational DL models on radio signals. Adopting a Convolutional LSTM architecture for efficient spatio-temporal processing, we pretrain the model with an unlabelled radio dataset collected from over-the-air measurements. Subsequently, the pretrained model is fine-tuned for two downstream tasks: spectrum forecasting and segmentation. Experimental results demonstrate that our methodology achieves competitive performance in both forecasting accuracy and segmentation, validating its effectiveness for developing foundational radio models. |
| title | Self-Supervised Radio Pre-training: Toward Foundational Models for Spectrogram Learning |
| topic | Signal Processing Artificial Intelligence Machine Learning Networking and Internet Architecture |
| url | https://arxiv.org/abs/2411.09849 |