Scaling Law in Neural Data: Non-Invasive Speech Decoding with 175 Hours of EEG Data
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910521439027200 |
|---|---|
| author | Sato, Motoshige Tomeoka, Kenichi Horiguchi, Ilya Arulkumaran, Kai Kanai, Ryota Sasai, Shuntaro |
| author_facet | Sato, Motoshige Tomeoka, Kenichi Horiguchi, Ilya Arulkumaran, Kai Kanai, Ryota Sasai, Shuntaro |
| contents | Brain-computer interfaces (BCIs) hold great potential for aiding individuals with speech impairments. Utilizing electroencephalography (EEG) to decode speech is particularly promising due to its non-invasive nature. However, recordings are typically short, and the high variability in EEG data has led researchers to focus on classification tasks with a few dozen classes. To assess its practical applicability for speech neuroprostheses, we investigate the relationship between the size of EEG data and decoding accuracy in the open vocabulary setting. We collected extensive EEG data from a single participant (175 hours) and conducted zero-shot speech segment classification using self-supervised representation learning. The model trained on the entire dataset achieved a top-1 accuracy of 48\% and a top-10 accuracy of 76\%, while mitigating the effects of myopotential artifacts. Conversely, when the data was limited to the typical amount used in practice ($\sim$10 hours), the top-1 accuracy dropped to 2.5\%, revealing a significant scaling effect. Additionally, as the amount of training data increased, the EEG latent representation progressively exhibited clearer temporal structures of spoken phrases. This indicates that the decoder can recognize speech segments in a data-driven manner without explicit measurements of word recognition. This research marks a significant step towards the practical realization of EEG-based speech BCIs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_07595 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Scaling Law in Neural Data: Non-Invasive Speech Decoding with 175 Hours of EEG Data Sato, Motoshige Tomeoka, Kenichi Horiguchi, Ilya Arulkumaran, Kai Kanai, Ryota Sasai, Shuntaro Neurons and Cognition Human-Computer Interaction Sound Audio and Speech Processing Brain-computer interfaces (BCIs) hold great potential for aiding individuals with speech impairments. Utilizing electroencephalography (EEG) to decode speech is particularly promising due to its non-invasive nature. However, recordings are typically short, and the high variability in EEG data has led researchers to focus on classification tasks with a few dozen classes. To assess its practical applicability for speech neuroprostheses, we investigate the relationship between the size of EEG data and decoding accuracy in the open vocabulary setting. We collected extensive EEG data from a single participant (175 hours) and conducted zero-shot speech segment classification using self-supervised representation learning. The model trained on the entire dataset achieved a top-1 accuracy of 48\% and a top-10 accuracy of 76\%, while mitigating the effects of myopotential artifacts. Conversely, when the data was limited to the typical amount used in practice ($\sim$10 hours), the top-1 accuracy dropped to 2.5\%, revealing a significant scaling effect. Additionally, as the amount of training data increased, the EEG latent representation progressively exhibited clearer temporal structures of spoken phrases. This indicates that the decoder can recognize speech segments in a data-driven manner without explicit measurements of word recognition. This research marks a significant step towards the practical realization of EEG-based speech BCIs. |
| title | Scaling Law in Neural Data: Non-Invasive Speech Decoding with 175 Hours of EEG Data |
| topic | Neurons and Cognition Human-Computer Interaction Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2407.07595 |