Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2507.22094 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913965271941120 |
|---|---|
| author | Mehlman, Nicholas Gagnon-Audet, Jean-Christophe Shvartsman, Michael Niu, Kelvin Miller, Alexander H. Sodhani, Shagun |
| author_facet | Mehlman, Nicholas Gagnon-Audet, Jean-Christophe Shvartsman, Michael Niu, Kelvin Miller, Alexander H. Sodhani, Shagun |
| contents | Surface electromyography (sEMG) signals offer a promising avenue for developing innovative human-computer interfaces by providing insights into muscular activity. However, the limited volume of training data and computational constraints during deployment have restricted the investigation of scaling up the model size for solving sEMG tasks. In this paper, we demonstrate that vanilla transformer models can be effectively scaled up on sEMG data and yield improved cross-user performance up to 110M parameters, surpassing the model size regime investigated in other sEMG research (usually <10M parameters). We show that >100M-parameter models can be effectively distilled into models 50x smaller with minimal loss of performance (<1.5% absolute). This results in efficient and expressive models suitable for complex real-time sEMG tasks in real-world environments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_22094 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Scaling and Distilling Transformer Models for sEMG Mehlman, Nicholas Gagnon-Audet, Jean-Christophe Shvartsman, Michael Niu, Kelvin Miller, Alexander H. Sodhani, Shagun Audio and Speech Processing Artificial Intelligence Human-Computer Interaction Machine Learning Surface electromyography (sEMG) signals offer a promising avenue for developing innovative human-computer interfaces by providing insights into muscular activity. However, the limited volume of training data and computational constraints during deployment have restricted the investigation of scaling up the model size for solving sEMG tasks. In this paper, we demonstrate that vanilla transformer models can be effectively scaled up on sEMG data and yield improved cross-user performance up to 110M parameters, surpassing the model size regime investigated in other sEMG research (usually <10M parameters). We show that >100M-parameter models can be effectively distilled into models 50x smaller with minimal loss of performance (<1.5% absolute). This results in efficient and expressive models suitable for complex real-time sEMG tasks in real-world environments. |
| title | Scaling and Distilling Transformer Models for sEMG |
| topic | Audio and Speech Processing Artificial Intelligence Human-Computer Interaction Machine Learning |
| url | https://arxiv.org/abs/2507.22094 |