IMG2IMU: Translating Knowledge from Large-Scale Images to IMU Sensing Applications
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2022
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911785876979712 |
|---|---|
| author | Yoon, Hyungjun Cha, Hyeongheon Nguyen, Hoang C. Gong, Taesik Lee, Sung-Ju |
| author_facet | Yoon, Hyungjun Cha, Hyeongheon Nguyen, Hoang C. Gong, Taesik Lee, Sung-Ju |
| contents | Pre-training representations acquired via self-supervised learning could achieve high accuracy on even tasks with small training data. Unlike in vision and natural language processing domains, pre-training for IMU-based applications is challenging, as there are few public datasets with sufficient size and diversity to learn generalizable representations. To overcome this problem, we propose IMG2IMU that adapts pre-trained representation from large-scale images to diverse IMU sensing tasks. We convert the sensor data into visually interpretable spectrograms for the model to utilize the knowledge gained from vision. We further present a sensor-aware pre-training method for images that enables models to acquire particularly impactful knowledge for IMU sensing applications. This involves using contrastive learning on our augmentation set customized for the properties of sensor data. Our evaluation with four different IMU sensing tasks shows that IMG2IMU outperforms the baselines pre-trained on sensor data by an average of 9.6%p F1-score, illustrating that vision knowledge can be usefully incorporated into IMU sensing applications where only limited training data is available. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2209_00945 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | IMG2IMU: Translating Knowledge from Large-Scale Images to IMU Sensing Applications Yoon, Hyungjun Cha, Hyeongheon Nguyen, Hoang C. Gong, Taesik Lee, Sung-Ju Machine Learning 68T20 Pre-training representations acquired via self-supervised learning could achieve high accuracy on even tasks with small training data. Unlike in vision and natural language processing domains, pre-training for IMU-based applications is challenging, as there are few public datasets with sufficient size and diversity to learn generalizable representations. To overcome this problem, we propose IMG2IMU that adapts pre-trained representation from large-scale images to diverse IMU sensing tasks. We convert the sensor data into visually interpretable spectrograms for the model to utilize the knowledge gained from vision. We further present a sensor-aware pre-training method for images that enables models to acquire particularly impactful knowledge for IMU sensing applications. This involves using contrastive learning on our augmentation set customized for the properties of sensor data. Our evaluation with four different IMU sensing tasks shows that IMG2IMU outperforms the baselines pre-trained on sensor data by an average of 9.6%p F1-score, illustrating that vision knowledge can be usefully incorporated into IMU sensing applications where only limited training data is available. |
| title | IMG2IMU: Translating Knowledge from Large-Scale Images to IMU Sensing Applications |
| topic | Machine Learning 68T20 |
| url | https://arxiv.org/abs/2209.00945 |